Method for producing a color image and imaging device employing same

ABSTRACT

A method of producing a color image using a display comprised of pixels comprising red, green and blue primary color subpixels. The method comprises reducing the color gamut and increasing the brightness of the image relative to a base level, decreasing power to the display to reduce the brightness of the image, restoring color to the image to approximately the base level by modifying image pixel data using a three-dimensional lookup table to produce output image pixel data, and communicating the output image pixel data to the display. The display may be an LCD display, an LED display, an OLED display, a plasma display, and a DMD projector. Reducing the color gamut and increasing the brightness of the image may be accomplished by adding white to the image. The white may be added adaptively according to an algorithm by which the amount of white added decreases with increasing color saturation.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 12/869,624, filed Aug. 26, 2010, which claims priority fromU.S. Provisional Patent Application No. 61/238,706 filed Sep. 1, 2009,the disclosures of which are incorporated herein by reference.

BACKGROUND

1. Technical Field

Processing and projection or display of color images on surfaces, ontelevisions, on game displays, on computers or by other electronicdisplay media. In particular, methods and systems for display powersavings, and extending battery life in color image display devices bymanaging color output while not degrading color image quality.

2. Description of Related Art

The projection and/or display of color images is an active area ofcommercial research and development. New image display, television,games, computers and projection products and viewing experiences arebeing launched in the marketplace on a regular basis. In one aspect ofthe marketplace, digital cinema or video projector technology thatutilizes colored light emitting diodes (LEDs) as the source of theprimary colors for imaging, offers the promise of extreme, wide colorgamut along with very long life, low heat illumination. LED brightnessis currently limited, however, requiring three optical systems and threeimage modulators, i.e., one for each of the red, green, and blue (RGB)color channels, for the brightest images. Current projector lamptechnology is of higher brightness and can take advantage of singleoptical systems and single image modulators using complex color filterwheels to provide full color display. In a second aspect of themarketplace, televisions, game displays and computer displays such asliquid crystal displays (LCDs) are now being introduced with LEDs as thebacklit light source to again take advantage of the extreme, wide colorgamut, long life and low heat output of LEDs. In a third aspect of themarketplace, projectors, televisions, game displays and computerdisplays are being introduced with more than the typical three (RGB)colors to improve brightness and expand the color gamut. Such productsoffer the promise and technical challenge of how to best use the widecolor gamut.

In a color image projector, in order to gain the advantage of theavailable wide color gamut, longer life, and lower heat of LEDillumination, and to achieve maximum brightness with a single opticalsystem and single image modulator, the multiple RGB channels may becombined for some portion of time during image frames to produce a whiteexposure during a pixel, or a portion of an image pixel area can includea clear filter area that transmits all of the LCD backlight spectrum fora 4^(th) white sub-pixel. Adding these multiple RGB channels during animage frame duty cycle or area will increase the brightness, but willalso reduce the colorfulness by desaturating the pure RGB colors.

Furthermore, in prior art projectors, color rendering is accomplished byprocessing each of the RGB channels independently with matrix operatorsor with one-dimensional color look-up tables. In some projectors, theRGB colors and the combinations of two and three colors may beindependently controlled. However, such control does not provide fullthree-dimensional color processing. With these limited processingoptions, it is not possible to display images optimally in human visualsystem (HVS) perceptual terms. For example, it is not possible to rendervisual lightness contrast without affecting either or both of hue andchroma. Achieving optimal visual processing that provides the brightest,most colorful images, while preserving perceived color accuracy requiresthree-dimensional color processing.

In providing any color image for viewing by a human observer, whether itis an image printed on a substrate, an electronic display, television,or a projection onto a viewing surface, the perception of color stimuliby the human observer is dependent upon a number of factors. In theInternational Lighting Vocabulary published in 1987 by the CommissionInternationale de l'éclairage (CIE), it is noted as follows: “Perceivedcolor depends upon the spectral distribution of the color stimulus, onthe size, shape, structure, and surround of the stimulus area, on thestate of adaptation of the observer's visual system, and on theobserver's experience of the prevailing and similar situations ofobservations.”

Moreover, in a treatise on the stained glassed windows at the cathedralat Chartres, The Radiance of Chartres: Studies in the Early StainedGlass of the Cathedral, (Columbia University Studies in Art History andArchaeology, No. 4), Random House, 1^(st) Ed., 1965, author James RosserJohnson wrote that, “ . . . the experience of seeing these windows . . .is a very complicated experience . . . that spans many aspects ofperception.” Yet fundamentally, “ . . . when the spectator enters theCathedral from the bright sunlight, . . . the visitor must step withcaution until his eyes have made a partial dark adaptation . . . thenthe details of the interior will seem lighter and clearer while, at thesame time, the [stained-glass] windows become richer and more intense.”

Adaptation plays a powerful role in the instance depicted in Johnson'snarrative. By adapting to the darkness or lower, perceived diffuse whiteof the cathedral's interior, the colors of the windows appearexceedingly brilliant, invoking a perception, in the words of VincentScully, Architecture, The Natural and Manmade, St. Martin's Press, 1991,that, “ . . . transcend[s] the statics of the building masses, therealities of this world . . . [creating] a world of illusion, shaped byand for the heavenly light of the enormous stained glass windows.” Whilesuch a perceptual experience is certainly complex and affected by themany characteristics of the human visual system (HVS), the richness ofit is largely and simply made possible by the broad extent ofsensitivity of the HVS and its innate ability to adapt to its surround.

The HVS is capable of adapting to an incredible range of luminance. Forexample, the HVS may adapt its light sensitivity over a range of abouteight orders of magnitude, e.g., from a starlit, moonlit night having aluminance of about 0.0001 candela per square meter (cd/m2) to a brightlylit summer day of about 600 to 10,000 cd/m2. Equally remarkable is thatthe HVS may accommodate over five orders of magnitude of luminance atany given instant for the perception of complex visual fields that areroutinely experienced. This adaptation occurs relative to diffuse white,i.e., an area in the scene that appears white. The perceptions oflightness and chroma are then relative to this white. The higher thebrightness of the perceived white, the lower the brightness and chromaof similarly illuminated objects in the scene will appear to theobserver; conversely, the lower its brightness, the brighter and morecolorful such objects appear.

This means that changing the stimulus that appears white affects theappearance of all other stimuli in the scene. For a display orprojection of an image, these powers of adaptation can be harnessed toexpand the gamut of the medium in the perceptual sense. For any imagedisplay, and particularly single modulation LED displays such as thoseemploying a digital micromirror device (DMD), the projected image can bemade to appear brighter by the addition of light from combining RGBcolors for some portion of the image frame time. In so doing, the powersof HVS adaptation are exploited to increase the apparent brightness andlightness contrast of the displayed images. For displays illuminated byred, green, and blue LEDs, although the added light reduces the actualdisplay color gamut provided by the “LED primaries,” the R, G, and Bprimary colors of the LEDs often exceed the current video standards,such as e.g., ITU Radiocommunication Sector (ITU-R) RecommendationBT.709, which is the United States standard for the format ofhigh-definition television and consumer digital media. Thus some colorswhich are possible to output by the R, G, and B LEDs, or displays withmore than three colors and extended color gamut are not available to beencoded in the input color data for display in accordance with suchstandards. Optimal use of these extended colors requires fullthree-dimensional color processing and can be further optimized usingknowledge of the HVS. Prior attempts to process the current videostandards, such as with one-dimensional color processing and colormatrices, or without use of HVS models have resulted in unsatisfactoryand unrealistic displayed images and high rates of product return byconsumers.

Illustrative of some of these attempts, FIGS. 1A-1D are two-dimensionalschematic diagrams of various prior art ways for processing input colordata to produce output color data for rendering a color image. FIG. 1Ashows a color hue/saturation/contrast/brightness method, depicting theglobal controls that rotate hue, stretch saturation and contrast andraise brightness. All colors are changed with these controls with no wayto isolate a given color or color region like flesh tones.R_(in)/G_(in)/W_(in) are input HD709 standard colors, andR_(out)/G_(out)/W_(out) are more pure output LED Colors. There are fourcontrols, and if each control is provided with 20 settings for example,there are 80 global choices.

FIG. 1B shows a color matrix method depicting a linear matrix globalcontrol that rotates and scales the color axes. All colors are changedglobally with no way to isolate local colors like flesh tones.R_(in)/G_(in)/W_(in) are input HD709 standard colors, andR_(out)/G_(out)/W_(out) are more pure output LED colors. If a 3×3 matrixis used, there are nine global choices.

FIG. 1C shows a color gamma tables method depicting gamma globalcontrols that independently maps each input color non-linearly to dothings such as increase contrast. It can be seen that, e.g., red changesare the same for all green values. The same relationships occur withother combinations of primary colors. Thus gamma controls are global,with no way to locally isolate colors, such as flesh tones.R_(in)/G_(in)/W_(in) are input HD709 standard colors, andR_(out)/G_(out)/W_(out) are more pure output LED colors. With threeprimary colors having 4096 settings, there are 12288 global choices.

FIG. 1D shows a 2D example of an RGBCYMW seven color mapping method. Inthis simple example of 7-color tetrahedral processing, the RBG/RGWtriangles are independently processed using linear interpolation ofinput/output control values at each vertices. This is a global control,with no way to isolate local colors or regions like flesh tones.R_(in)/G_(in)/W_(in) are input HD709 standard colors, andR_(out)/G_(out)/W_(out) are more pure output LED colors. With 14 In/Outcolors, there are 14 global choices. R_(in)/G_(in)/W_(in) are inputHD709 standard colors, and R_(out)/G_(out)/W_(out) are more pure outputLED Colors.

Digital Cinema Initiatives, LLC (DCI) is a joint venture of major motionpicture studios, which was formed in 2002 to create standards fordigital cinema systems, including image capture and projection. Thedigital color standard adopted by the studios for professional moviereleases in the DCI format is 12 bits per primary color, nonlinear CIEXYZ Tristimulus values. This is the first time that a digital standardhas been established that is encoded in visual color space and thereforeindependent of any imaging device. For example, using this standard, thesame digital file can be displayed to produce the specified color on atelevision or a printer. The color gamut of this digital color standardis larger than any possible display.

FIG. 3 is a diagram of color gamuts, including color gamuts of the DCIand HD709 standards, and color gamuts of various media and/or imagingdevices. It can be seen that in diagram 400, the color gamuts 406, 408,410, and 412 of the various imaging devices are substantially largerthan the HD709 standard 404. Accordingly, to take full advantage of thecolor capabilities of these imaging devices 406-412, the color gamut ofthe HD709 standard must be mapped upwardly, to render the full colors ofthe larger color gamut, while simultaneously preserving flesh tones andother memory colors, and optimizing the particular device for viewing ina particular environment.

It can also be seen that the large triangular boundary 402 thatrepresents the DCI standard encompasses all of the color gamuts of themedia and/or imaging devices, as well as the color gamut of the HD709standard 404. Accordingly, the digital color standard input color gamut402 must be contracted or reduced to fit within the color gamut of aphysical display such as a television or projector. Truncating orclipping those input digital color values of the DCI standard that lieoutside of the color gamut boundary of the display device will causeloss of color saturation and detail and create a visually sub-optimaldisplayed image. Conventional video processing using one-dimensionalcolor tables and linear matrices will also produce sub-optimal displayedimages. Optimal display of these contracted colors requires fullthree-dimensional color processing and can be further optimized usingknowledge of the HVS and the state of visual adaptation in particularviewing environments.

Also, image and video media display products are now being reduced insize. Examples of such products are the new miniature pico-projectorsand portable, handheld displays such as iPods® or iPads®. Because ofpower, heat, and size limitations, these displays generally have reducedcolor gamuts due to reduced contrast or reduced color saturation. Theyalso are often used in widely differing viewing environments bothindoors and outdoors. Improvement of the overall quality of thesesmaller gamut displays with conventional image and video input iscritical to product value. Conventional video processing usingone-dimensional color tables and linear matrices will also producesub-optimal displayed images. Optimal display of these contracted colorsrequires full three-dimensional color processing and can be furtheroptimized using knowledge of the HVS and the state of visual adaptationin particular viewing environments.

Additionally, the capabilities of HVS adaptation are affected by theviewing environment. In a dark room, higher contrast is needed in aprojected or displayed image for an equally perceived viewing experienceas compared to a room with normal room lighting or viewing the sameimage in bright outdoor lighting. Relative to bright outdoor lighting,the HVS adaptation to the dark room and the lower overall imagebrightness combine to reduce the perceived image contrast. In a brightlylit room, less contrast is needed due to brightness adaptation and morecontrast is needed due to viewing flare from room lights illuminatingthe dark areas of the displayed image.

In image displays, televisions, and/or projectors using high brightnesslight sources or expanded or reduced color gamuts, there is therefore aneed in displaying and/or projecting images to optimize the increase inperceived brightness, contrast, and colorfulness while preservingexpected memory colors of the displayed image such as flesh tones. Suchan optimization should take into account that not all colors should beadjusted in the same manner and to the same extent. To do so wouldresult in images containing certain details that appear unsatisfactoryto a human observer. For example, if a flesh tone of a face in an imageis modified in the same manner as a relatively saturated color ofanother object in the image, the face will be perceived as “pink,”“orange,” or “burnt” by an observer and thus will be perceived asunsatisfactory. There is therefore a need to achieve this optimizationwhile also preserving certain known colors, such as flesh tones, greytones, named colors (such as commercial “brand” colors), and other“memory” colors in the image. Prior attempts to process the video inputswith one-dimensional color processing and color matrices for suchextended brightness, contrast or color gamut displays, have resulted inunsatisfactory and unrealistic displayed images and high rates ofproduct return by consumers.

Current projectors, televisions or displays that attempt to enhance orimprove perceived color quality with processing that is in any waydifferent than exact colorimetric color reproduction, do not preservememory colors in the background. A memory color may be characterized asa localized volume in a color space, as will be described subsequentlyherein. The algorithms used in current image displays, televisions andprojectors cannot uniquely preserve a volume within a three-dimensionalcolor space while changing a different volume within the samethree-dimensional color space using one dimensional tables, or matrices,or enhancements which are applied to all colors in the 3D space. Forexample, in some image projectors, color enhancement is attempted usingoutput color definitions of the seven input colors RGBCMYW(red-green-blue-cyan-magenta-yellow-white). This may allow one toprovide a bright white in an image without changing red, for example,but it does not allow one to specify any point or localized volume of amemory color in a 3D color space, which is required to preserve thatmemory color. As a result, when current image displays, televisions andprojectors provide enhanced colors, they do so across the entire colorgamut, “enhancing” certain memory colors such as flesh tones such that atypical human observer finds them unsatisfactory and not perceptuallyoptimal. In such image devices, the color enhancement is somewhatarbitrary; it does not preserve memory colors, nor produce a perceiveddisplay image that is realistic for a better viewing environment.

More generally, to the best of the applicants' knowledge, no one hasimplemented the use of three dimensional color tables in 3D colorprocessing to improve image quality for video images, or in 3D colorprocessing for gamut mapping to larger color gamut displays than aparticular image standard, or in gamut mapping to smaller color gamutdisplays than a particular image standard, or in 3D color mapping todisplays with secondary color capability and more than three colors thatare primary or secondary, using visual models of the human visual systemor otherwise. Currently, standard color processing for displays uses onedimensional tables, 3×3 matrices or matrix mathematics that allowsoutput definition of a small number of colors like RGBCYMW.

3D color tables have been implemented for color calibration, but in suchcircumstances, the tables are small (e.g., 7×7×7). These 3Dlook-up-tables are used instead of one dimensional tables and 3×3matrices because the small 3D look-up-tables are generally faster,albeit at the expense of some loss of precision. In any case,significant color improvement or enhancements to deliver color “looks,”or gamut mapping or mapping to displays with secondary or more thanthree primary colors with such small tables is not possible.

Another problem in certain types of image rendering devices is that theoutputs of the primary color light sources are not stable. This isparticularly true for image rendering devices that use organic lightemitting diodes (OLEDs) as the sources of the primary colors red, green,and blue. A known problem with OLED displays is that the blue OLEDtypically has had a considerably shorter lifespan than the red and greenOLEDs. One measure of OLED life is the decrease of luminance to half thevalue of original brightness. The luminance of currently available blueOLEDs decreases to half brightness in a much shorter time than the redor green OLEDs. During the operation of an OLED display, thisdifferential color change between the blue OLED and the red and greenOLEDs changes the color balance of the display. This change is much moreobjectionable to a viewer than a decrease in overall brightness of thedisplay.

To the best of the applicants' knowledge, the problem of managing theoverall lifespan of OLED displays has not be solved adequately, whichhas led to significant delays in product introduction in themarketplace. There is therefore a need to provide a solution thatmanages the overall quality and lifespan of the relative luminances ofthe red, green and blue OLEDs in a display device.

Another problem in certain types of image rendering devices is that thebattery life of the device is not sufficient to satisfy users' needs. Asadvances have occurred in wireless communication technology, consumersare spending increasing amounts of time using such devices, while alsodemanding increased quality in the color images that they display.Mobile displays are being used increasingly for all types ofentertainment media including long form media such as television andmovies. These uses occur in diverse lighting environments, and the usagelifetime is a critical characteristic to consumers. In a typicalhand-held device, both the increase in the duty cycle and the demand formaximum perceived image quality of the display have placed a heavyburden on the batteries used to power such devices.

Consumers are often finding that the battery life of a particular deviceis less than stated in sales literature and/or owner's manuals. The alsofind that they cannot use their devices as often as they would like (insome cases nearly continuously over the course of a day), withoutcarrying some sort of a charging cord and plug that connects to a 12volt automobile jack, a 120 VAC outlet, or a USB port. This is asignificant annoyance, and device makers recognize the overall problem.

One solution would be to increase the overall size of the batteries inthe device. However, this is clearly unsatisfactory, because spacewithin any of these devices is at a premium, and manufacturers areunwilling to allocate additional volume to batteries when it is neededfor many other purposes of equal or greater priority.

Another measure to increase battery life is to reduce the powerdelivered to the image display, which may be a plasma display, a liquidcrystal display (LCD), or an organic light emitting diode (OLED)display. This generally results in a reduction in the brightness of thedisplay, which is also unsatisfactory to consumers, particularly whenviewing the display in full daylight. Their expectations of imagequality are continuously being raised. For example, a recent iPad®product of Apple launched in Mar. 2012 offers “breakthrough technology”in its display, as set forth at http://www.apple.com/ipad/features/:“The Retina display on the new iPad features a 2048-by-1536 resolution,44 percent greater color saturation, and an astounding 3.1 millionpixels—in the same 9.7-inch space. That's four times the number ofpixels in iPad 2 and a million more than an HDTV.” In general, thenewest, most desirable image displays on the market are typicallydemanding more battery power, rather than less.

Consumers are unwilling to trade battery life for image quality. Therecontinues to be an increasing need for color display devices that canprovide the highest of image quality while also providing sufficientlylong battery life to meet consumer demands. Products which can meet thisneed will have a significant competitive advantage in the marketplace.

SUMMARY

A color-enhanced image display, television, or projection that maintainscertain known colors and optimizes colorfulness and contrast will havethe highest visual perceptual quality if and only if the rendering isaccomplished wherein the input RGB colors are processedinter-dependently. This requires the use of a three-dimensional colorlook-up table, also referred to herein as a 3D LUT. The colorenhancement may entail increased brightness and/or a larger or smallercolor gamut, depending upon the particular image display or projector.In prior art image displays and projectors in which traditional matricesand one dimensional color tables operate independently on the RGB inputcolors, a brighter display is not possible without affecting hue. Forexample, blue skies will be shifted towards purple, flesh tones will bealtered in unpredictable ways, and many other color artifacts may bepresent, depending upon the content of the particulardisplayed/projected image. The use of 3D color look-up tables enablesbrighter, higher contrast, and more colorful image displays andprojections without color artifacts. Using methods of the presentinvention, this can be accomplished for image displays or projectorswhich have color gamuts about the same as that of a given colorstandard, or larger than the standard, or smaller than the standard. Thecolor rendering of such image displays or projectors can be enhancedusing three dimensional tables with differing methods in each volume andwith visual models.

In one aspect of the invention, a first method of producing a colorimage is provided comprising providing input image data from an imagesource such as a camera; generating an at least three-dimensionallook-up table of values of input colors and output colors, wherein thevalues in the lookup table convert the input image color data to outputimage color data in an image rendering unit; loading the at leastthree-dimensional look-up table into an image color renderingcontroller; loading the input image data into the imaging colorrendering controller; processing the input image data through the atleast three-dimensional look-up table to produce output color valuesstored at the addresses in the at least three-dimensional look-up table;and outputting the output color values to the image rendering unit toproduce an output image that is perceived to have at least one ofenhanced brightness, enhanced contrast, and enhanced colorfulnesscompared to the input image.

The values in the lookup table may be calculated based upon a visualmodel of the human visual system and they may include modeling toimprove the perceived brightness or contrast or colorfulness fordifferent viewing environments. The at least one of enhanced brightness,enhanced contrast, or enhanced colorfulness introduced by the at leastthree dimensional look-up-table may produce a chosen artistic perceptionin the output image. The image rendering unit may have an expanded colorgamut greater than the color gamut of the input image data, wherein theoutput colors to the image rendering unit utilize the expanded colorgamut, or the image rendering unit may have a reduced color gamutsmaller than the color gamut of the input image data, wherein the outputcolors to the image rendering unit utilize the smaller color gamut. Theinput image data may contain memory colors and non-memory colors, andthe method may include identifying the memory colors in the input imagedata to be substantially maintained, characterizing the memory colorsand non-memory colors with respect to their chromaticities, andproducing an image with substantially maintained memory colors using theimage rendering unit. In such circumstances, the perceived colorfulness,brightness, and contrast of the non-memory colors are changeddifferently than perceived colorfulness, brightness, and contrast of thememory colors. They may be increased more than perceived colorfulness,brightness, and contrast of the memory colors. In one embodiment, theperceived colorfulness, brightness, and contrast of the non-memorycolors are increased more than perceived colorfulness, brightness, andcontrast of the memory colors. Generating the at least three-dimensionallook-up table may include computing enhanced lightness, chroma, and huefor the memory colors using a non-linear enhancement function. Theenhancement function may be a sigmoidal function. More than one at leastthree-dimensional look-up table for the color transformation of thenon-memory colors and the memory colors may be generated and used. Eachof the at least three dimensional look-up tables may be optimized for adifferent viewing environment of the image rendering unit. The methodmay further include providing a sensor for measuring the ambient lightin the viewing environment.

The input image data may be of a first color standard, and the methodmay further include converting the input image data of the first inputcolor standard into an input color specification for inputting into thethree-dimensional look-up table. The at least three-dimensional look-uptable may have at least three input colors and/or at least three outputcolors. The at least three output colors may be any combination ofprimary colors as independent light sources or secondary colors definedas combinations of primary colors. The at least three dimensionallook-up table may be losslessly compressed to reduce storage use inmemory of the image color rendering controller. The method may furtherinclude calibrating the image rendering unit by measuring the colorresponse of the image rendering unit, and then modifying the outputimage data either by additional processing after the at leastthree-dimensional look-up-table or by including the required calibrationin the at least three-dimensional look-up-table.

The image color rendering controller may be contained within the imagerendering unit, or it may be external to the image rendering unit. Anauxiliary imaging device controller may be in communication with theimage color rendering controller and the image rendering unit. The imagerendering unit may be selected from, but not limited to a projector, atelevision, a computer display, and a game display, and may use DMD,plasma, liquid crystal, liquid crystal-on-silicon modulation, or directmodulation of the light source. The light source may be an LED, OLED,laser, or lamp light sources. Without limitation, the image colorrendering controller may be in communication with at least one of acable TV set-top box, a video game console, a personal computer, acomputer graphics card, a DVD player, a Blu-ray player, a broadcaststation, an antenna, a satellite, a broadcast receiver and processor,and a digital cinema.

The image rendering unit may include an algorithm for colormodification, wherein the at least three-dimensional look-up tablefurther comprises processing the input image data to compensate for thecolor modification performed by the image rendering unit. The imagerendering unit may include an algorithm for creating secondary colorsfrom primary colors, and the at least three-dimensional look-up tablefurther comprises compensating for the color modification performed bythe addition of the secondary colors in the image rendering unit.

The at least three-dimensional look-up table may further includeprocessing the input image data to increase perceived color, brightness,and contrast to compensate for the reduction in perceived color,brightness, and contrast caused by the algorithm for color modificationin the image rendering unit. The at least three-dimensional look-uptable may contain a transformation from a suboptimal viewing environmentto an improved viewing environment including the visual adaptation ofthe human visual system. The at least three-dimensional look-up tablemay include the definition of secondary colors, and may further containenhanced lightness, chroma, and hues to increase perceived colorfulness,contrast, or brightness to compensate for the loss in perceivedcolorfulness, contrast, or brightness due to addition of the secondarycolors by the image rendering unit. The at least three-dimensionallook-up table may further include processing the input image data toinclude chromatic adaptation of the human visual system to a specifiedwhite point that increases the brightness of the image rendering unit.

The instant method may be used in the display or projection of twodimensional (2D) or “three dimensional” (3D) images. The 3D images aretypically produced by providing 2D stereo images simultaneously or inrapid sequence taken from two perspectives, so as to provide theobserver with the illusion of depth perception. The image rendering unitmay be a “3D” unit. By way of illustration, and not limitation, the unitmay be e.g., an autostereoscopic display, or it may include a polarizingfilter to separate the 2D stereo images being projected and directed tothe eyes of an observer using polarization glasses, or it may include ashuttering mechanism to separate the 2D stereo images being projectedand directed to the eyes of an observer using time synced shutterglasses. In any case, both sets of 2D images may be processed accordingto the instant method to deliver 3D images that are perceived by anobserver to have enhanced brightness, and/or enhanced contrast, and/orenhanced colorfulness.

In another aspect of the invention, an additional method of producing acolor image is provided, the method comprising providing input imagedata of a first color gamut and an image rendering unit of a second,expanded or reduced color gamut; generating an at leastthree-dimensional look-up table of values of input colors and outputcolors, wherein the values in the lookup table expand or reduce theinput image data to encompass the second color gamut of the imagerendering unit; loading the at least three-dimensional look-up tableinto an image color rendering controller; loading the input image datainto the imaging color rendering controller; processing the input imagedata through the at least three-dimensional look-up table using theinput image data as addresses into the at least three-dimensionallook-up table to produce output image data from the output color valuesstored at the addresses in the at least three-dimensional look-up table;and outputting the output image data to the image rendering unit toproduce an output image that is perceived to have at least one ofenhanced brightness, enhanced contrast, and enhanced colorfulnesscompared to the input image. This method may also include the variousaspects and/or steps described above for the first method.

In another aspect of the invention, the models may include visual modelsof HVS perceptual adaptation to produce a projected or displayed imagethat appears as it would in a more optimal, well lit viewingenvironment. The image processing may include correcting for low levellighting of the surrounding environment and/or indoor or outdoor ambientlight added to the displayed image. More specifically, a method ofproducing a color image by an image rendering unit in a sub-optimalviewing environment is provided, the method comprising generating an atleast three-dimensional look-up table of values of input colors andoutput colors, the table containing a transformation from a suboptimalviewing environment to an improved viewing environment; loading the atleast three-dimensional look-up table into an image color renderingcontroller; loading the input image data into the image color renderingcontroller; processing the input image data through the at leastthree-dimensional look-up table using the input image data as addressesinto the at least three-dimensional look-up table to produce outputimage data from the output color values stored at the addresses in theat least three-dimensional look-up table; and outputting the outputimage data to the image rendering unit. This method may further includethe various aspects and/or steps described above for the first method.The improved viewing environment may be such that an observer mayperceive the color image to have more color, contrast, or brightness.

In yet another aspect of the invention, a method of producing a colorimage by an image rendering unit is provided, the method comprisinggenerating an at least three-dimensional look-up table of values ofinput colors and output colors, the three-dimensional look-up tablecontaining the definition of secondary colors or more than three primarycolors; loading the at least three-dimensional look-up table into animage color rendering controller; loading the input image data into theimage color rendering controller; processing the input image datathrough the at least three-dimensional look-up table using the inputimage data as addresses into the at least three-dimensional look-uptable to produce output image data from the output color values storedat the addresses in the at least three-dimensional look-up table; andoutputting the output image data to the image rendering unit to producean output image that is perceived to have at least one of enhancedbrightness, enhanced contrast, and enhanced colorfulness compared to theinput image. This method may also include the various aspects and/orsteps described above for the first method.

The secondary colors or more than three primary colors may be explicitlydefined, or the secondary colors or more than three primary colorsimplied in the design of a three in by three out look-up table for twoconditions. In either instance, measured responses of the imagerendering unit may be used to define the three-dimensional look-uptable, or mathematics provided by a manufacturer of the image renderingunit may be used to define the three-dimensional look-up table.Alternatively, an open definition of how the secondary colors or morethan three primary colors are used may be provided. This method may alsoinclude the various aspects and/or steps described above for the firstmethod.

In another aspect of the invention, the problem of displaying orprojecting an image that is optimal in human visual perceptual termsregardless of the ambient light and background environment of the imageis solved by using visual models to enhance the perceived colorfulness,contrast, or brightness of the image, thereby improving the perceivedquality of the image. The visual models of human visual perception maybe used to create look-up tables of at least three dimensions to processthe image to be displayed. Memory colors of the image may be preserved.The method may further include performing empirical visual studies todetermine the dependence of the preference of colorfulness, contrast, orbrightness on the ethnicities of the human observers, and defining theperceived quality of the image for each nationality of human observers.The method may further include adjusting the colorfulness, contrast, orbrightness of the image based upon one of the ethnicities of the humanobservers. The method may further include generating an at leastthree-dimensional look-up table of values of input colors and outputcolors, the three-dimensional look-up table adjusting the colorfulness,contrast, or brightness of the image to match the enhanced appearance ofanalog film systems or digital systems designed for cinemas. The methodmay further include adjusting the colorfulness, contrast, or brightnessof the image to produce a chosen artistic perception in the image.

In another aspect of the invention, a method of producing a color imageby an OLED display is provided that manages the overall quality andlifespan of the relative luminances of the red, green and blue OLEDs inthe display. The method comprises providing input image data andproviding the OLED display having at least three OLEDs, each OLED beingof a different primary color; generating an at least three-dimensionallook-up table of values of input colors and output colors, wherein thevalues in the lookup table convert the input image data to output imagecolor data of the OLED display in a manner that optimally manages thequality of the image and the lifetime of the at least three OLEDs;loading the at least three-dimensional look-up table into an image colorrendering controller; loading the input image data into the imagingcolor rendering controller; processing the input image data through theat least three-dimensional look-up table to produce output color valuesstored at the addresses in the at least three-dimensional look-up table;and outputting the output image data to produce the image by the OLEDdisplay. The values in the look-up table may be calculated based upon avisual model of the human visual system. This method may further includethe various aspects and/or steps described above for the first method.

The at least three OLEDs may be a red OLED, a green OLED, and a blueOLED. In such an instance, managing the quality of the image and thelifetime of the OLEDs may further include adding a white primary andmapping predetermined amounts of the grey component of RGB pixel valuesto the white primary to reduce the usage of RGB and extend the life ofthe red, green, and blue OLEDS. Alternatively, managing the quality ofthe image and the lifetime of the OLEDs may comprise adding otherprimary colors and mapping predetermined amounts of the RGB pixel valuesto the other primary colors to reduce the usage of RGB and extend thelife of the red, green, and blue OLEDS. The method may further compriseoperating the at least three OLEDs such that a first OLED does not reachend of life sooner than the other OLEDs, and the image quality of eachof the OLEDs is reduced about equally over time without perceivedartifacts or appearances predominantly of one of the OLED colors.

The method may be further comprised of having a controlled degradationof image quality due to changes in the outputs of at least one of theOLEDs, wherein the change of quality at any given point in time has theleast loss in perceived quality. The controlled degradation may betracked by accumulating and using usage data for all of the OLEDs. Thecontrolled degradation may be performed on the entire image over time,or on at least one portion of the image over time. The controlleddegradation may be performed by substantially maintaining the brightnessof the image while gradually reducing color saturation of the image overtime, or by reducing color saturation of the image to a greater extentin image pixels of low color saturation than in image pixels of highcolor saturation, or by substantially maintaining the brightness of theimage while reducing color saturation gradually using adaptive onedimensional tables on each of the primary colors.

The one dimensional tables on each primary color may be calculated usinga quality degradation model. The quality degradation model may averageamong one dimensional tables that are pre-designed to provide thetargeted image quality at specific OLED lifetimes. The one dimensionaltables may be produced by interpolation between a one dimensional tablefor when the OLEDs are initially operated and a one dimensional tablefor when the OLEDs are at the ends of their useful lifetimes.

In another aspect of the invention, in an image display, television, orprojector, the problem of achieving an expanded or maximum color gamutby temporally combining R, G, and B during an image frame duty cycle toincrease brightness while maintaining saturated pure R, G, and B colorsis solved by calculating the combinations of R, G, and B that maintain aphysical or perceived input color in a given viewing environment therebymaintaining physical or perceived color saturation and achievingincreased brightness. The calculated combinations are implemented in a3D look-up table.

In any of the above aspects of the invention, the color image to beproduced may contain “memory colors” as defined herein, and non-memorycolors. In general, the memory colors of the image that is produced arepreserved. The methods may include identifying the memory colors in theinput image data to be substantially maintained, characterizing thememory colors and non-memory colors with respect to their chromaticitiesin the image rendering unit, and producing an image comprising humanvisual system perceptually accurate memory colors using the imagerendering unit. The perceived colorfulness, brightness and contrast ofthe non-memory colors are increased more than perceived brightness andcontrast of the memory colors. In one embodiment, generating the atleast three-dimensional look-up table may include computing enhancedlightness, chroma, and hue for the memory colors using a sigmoidalenhancement function. More than one at least three-dimensional look-uptable may be generated for the color transformation of the non-memorycolors and the memory colors. Some or all of the at least threedimensional look-up tables may be optimized for a different viewingenvironment of the image rendering unit. In such an instance, the methodmay further include selecting one of the at least three-dimensionallook-up tables for loading into the image color rendering controllerbased upon the viewing environment of the image rendering unit. A sensormay be provided for measuring the ambient light in the viewingenvironment.

In a related aspect of the invention, the problem of displaying an imagethat simultaneously has high brightness and high colorfulness of amajority of colors (and particularly high saturation colors), whilemaintaining realistic “memory colors” is solved by adding white light orany combination of multiple R, G, B colors by combining R, G, and B forsome portion of the duty cycle of the image projection time, accordingto a 3D look-up table, which replaces the lost colorfulness of addingcolor combinations and at the same time preserves flesh tones and otherknown memory colors. The image data is processed with a 3D look-up tablein a manner that that increases the perceived colorfulness, brightness,and contrast while preserving flesh tones and other known memory colors.The 3D look-up table is created to produce the improved image quality.Visual models may be used to perform the image processing.

In any of the above aspects of the invention, the methods may furthercomprise converting the input image data of a first input color standardinto an input color specification for inputting into thethree-dimensional look-up table.

The solutions to the above problems may entail multi-dimensional look-uptables, with three dimensional look-up tables being one example. The atleast three dimensional lookup table may have three or more input colorsand three or more output colors. The output dimension may be differentfrom the input dimension, such as having RGBCYMW(red-green-blue-cyan-magenta-yellow-white) output values in an RGBtable, i.e. three values of input and seven values of output. The numberof outputs may also be greater than three due to the display having morethan three physical colors, i.e., more than three primary colors such asR, G, and B. In such an instance, the output colors could therefore bethe primary colors or combinations of the four or more colors. Ingeneral, the three or more than three output colors are any combinationof primary colors as independent light sources or secondary colorsdefined as combinations of primary colors. The at least threedimensional look-up table(s) may be losslessly compressed to reducestorage use in a memory of the image color rendering controller.

More specifically, according to the present disclosure, a method ofdisplaying an image containing memory colors and saturated colors isprovided comprising identifying the memory colors in input image data tobe substantially maintained, characterizing the memory colors withrespect to their chromaticities, and generating a three-dimensionallook-up table for a color transformation of saturated and memory colors.The three-dimensional look-up table is loaded into an imaging devicecontroller, and input image data is loaded into the imaging devicecontroller. The input image data is processed with an algorithm usingthe three-dimensional look-up table to produce output image data. Theoutput image data is output to an image rendering device, and a highbrightness, high contrast image comprising human visual systemperceptually accurate memory colors is displayed or projected.

In one embodiment, the method includes preprocessing, wherein onedimensional tables and matrices are provided for converting the varietyof possible input color standards into a preferred color input to the 3Dor higher dimensional color look-up-table. This is done for the purposeof making a single or reduced number of 3D or higher dimensional colorlook-up-tables adaptable to different video standards. In anotherembodiment, the algorithm containing the 3D or higher dimensionalmathematics is executed in real time by the central processing unit of acomputer in the image display or projection device so that the need fora 3D color table is obviated. This may be done if the device computer isprovided with adequate computational processing capability and memory.

In another embodiment, the method includes incorporating the variety ofpossible input color standards directly into the creation of the 3D orhigher dimensional color look-up-tables to adapt to different videostandards.

In some circumstances, the image rendering unit (such as, e.g., adisplay or projection device) is provided with some color modificationcapability that is “built in.” For example, the device may provided withan algorithm to add white or secondary colors, resulting in a loss ofcolorfulness, and a distortion in the appearance of memory colors. Insuch circumstances, the output values in the at least three-dimensionallook-up table are determined such that the input image data is processedto compensate for the color modification performed by the imagerendering unit. The method may thus include providing at least 3D colortables to adjust the color data in a manner that shifts it in adirection within the color space that compensates for the built in colormodification that is performed by the image rendering unit. The at leastthree-dimensional look-up table further comprises processing the inputimage data to increase perceived color, brightness, and contrast tocompensate for the reduction in perceived color, brightness, andcontrast caused by the algorithm for color modification in the imagerendering unit. In a more specific instance in which the image renderingunit includes an algorithm for creating secondary colors from primarycolors, the at least three-dimensional look-up table may furthercomprise compensating for the color modification performed by theaddition of the secondary colors in the image rendering unit. The valuesin the at least three dimensional lookup table may also be determinedsuch that the at least three-dimensional look-up table further comprisesprocessing the input image data to include chromatic adaptation of thehuman visual system to a specified white point that increases thebrightness of the image rendering unit. The at least three-dimensionallook-up table may also adjust the colorfulness, contrast, or brightnessof the image to be produced to match the enhanced appearance of analogfilm systems or digital systems designed for cinemas.

According to the present disclosure, there is further provided a devicefor producing a color image. The device is comprised of a computerincluding a central processing unit and a memory in communicationthrough a system bus. The memory may be a random access memory, or acomputer readable storage medium. The memory contains an at least threedimensional lookup table.

In one aspect of the invention, the at least three dimensional lookuptable contains values of input colors and output colors, wherein thevalues in the lookup table convert an input image color data set tooutput image color data in an image rendering unit that is connectableto the device.

In another aspect of the invention, the at least three dimensionallookup table may be produced by an algorithm for transforming inputimage data comprising memory colors and non-memory colors to a visualcolor space, and computing enhanced lightness, chroma, and hue for thememory colors and non-memory colors in the visual color space. Thealgorithm to produce the three dimensional lookup table may be containedin the memory.

In another aspect of the invention, the at least three dimensionallookup table includes values of input colors and output colors, whereinthe values in the lookup table convert a first color gamut of an inputimage data set to encompass a second expanded or reduced color gamut ofan image rendering unit that is connectable to the device.

In another aspect of the invention, the at least three dimensionallookup table contains a transformation from a suboptimal viewingenvironment to an improved viewing environment including the visual andchromatic adaptation of the human visual system.

In another aspect of the invention, the at least three dimensionallookup table contains the definition of secondary colors, and enhancedlightness, chroma, and hues to increase perceived colorfulness,contrast, or brightness to compensate for the loss in perceivedcolorfulness, contrast, or brightness due to addition of secondarycolors by an image rendering unit that is connectable to the device.

In another aspect of the invention wherein the image is perceived by ahuman observer, the memory may contain a visual model to enhance theperceived colorfulness, contrast, or brightness of the image.

In any of the above aspects of the invention, the device may furtherinclude the image rendering unit in communication with the computer. Theimage rendering unit may be selected from a projector, a television, acomputer display, and a game display, and may use DMD, plasma, liquidcrystal, liquid crystal-on-silicon modulation (LCOS), or directmodulation of the light source and LED, organic light emitting diode(OLED), laser, or lamp light sources. The device may further comprise anauxiliary imaging device including at least one of a cable TV set-topbox, a video game console, a personal computer, a computer graphicscard, a DVD player, a Blu-ray player, a broadcast station, an antenna, asatellite, a broadcast receiver and processor, and a digital cinema. Oneof a liquid crystal display, a plasma display, and a DMD projector maybe in communication with the auxiliary device. The device may furthercomprise a communication link to a source of input image data.

The at least three-dimensional look-up table includes the definition ofsecondary colors, and contains enhanced lightness, chroma, and hues toincrease perceived colorfulness, contrast, or brightness to compensatefor the loss in perceived colorfulness, contrast, or brightness due toaddition of the secondary colors by the image rendering unit.Alternatively or additionally, the at least three-dimensional look-uptable may contain a transformation from a suboptimal viewing environmentto an improved viewing environment including the visual and chromaticadaptation of the human visual system.

The memory of the device may contain a set of at least three dimensionallookup tables; each table of the set may be optimized for a differentviewing environment of the image rendering unit. The device may beprovided with a sensor for measuring the ambient light in the viewingenvironment.

In another aspect of the invention, a method is provided for displayinga high quality color image while reducing power consumption of thedisplay. The display may include a display screen, or the display mayproject the image onto a surface. The method may include providingdifferent amounts of added white or brightness to each pixel as afunction of the input pixel color value. Pixels with higher colorsaturation may have smaller amounts of white added so that the inputcolor saturation is better preserved, and pixels that are more neutralgrey can have more white added to increase the brightness more forblack-white pixels. This allows the input color saturation to bemaintained to a high degree, and thus the colorfulness (and thus theimage quality) perceived by an observer to be maintained. Additionally,if the amount of added white is reduced to a negligible amount for thehighest input color saturation, the full input color gamut of thedisplay can be maintained.

Various methods of reducing the added white with increasing colorsaturation are contemplated. For example, a Gaussian function of theinput color saturation may be used that has a one sigma (one standarddeviation) width that is smaller than the full input color gamut. Thismethod of adaptively adding white to a pixel results in the addedbrightness and corresponding power savings to be dependent on the image.For images that are more black-white, and thus low in color saturation,the power savings will be highest, and for highly colorful images, thepower savings will be the lowest. This type of processing may beimplemented in real time on image data so the image-dependent powersavings may thus change from image to image. This method is inaccordance with the color processing of this invention, because the lossin color saturation is in the mid-range of image color saturation andthe color processing of this invention has the highest color saturationrestoration in this range of input color saturation. Additionally,because most images have low-to-medium color saturation, and because thepower savings is greater for those types of image pixels, the resultingaverage power savings for a given set of image (such as a movie orvideo) is close to the maximum power savings for black-white images.

In another aspect of the invention, the problem of displaying highquality images on a portable display device over an extended period oftime is solved by modifying the primary colors of the display devicesuch that the resulting new primary colors are more efficient. Thisenables power to the device to be reduced, such as by using a lowerpower light source (for a liquid crystal display), or by using a lowerpower lamp or lower power white OLEDs. This results in less heatproduction and less other display management costs.

However, such a modification to the primary colors will change the colorgamut of the display, including its white point. Without correctiveaction, this can result in colors being rendered by the display that areunsatisfactory to the viewer. In a further aspect of the invention, thisproblem is solved by the use of the Applicants' three-dimensional colorlook-up tables (3DLUTs), which correct the colors to those that areaesthetically pleasing.

Thus the combination of modifying the primary colors of the displaydevice such that the resulting new primary colors are more efficient,and correcting the resulting shift in color gamut by the use ofthree-dimensional color look-up tables solves the problem of displayinghigh quality images on a portable display over an extended period oftime. By lowering power consumption, battery life of the portabledisplay is increased.

In accordance with the invention, there are multiple options for solvingthis overall problem. According to one aspect of the invention, there isprovided a method of producing a color image using a display comprisedof pixels comprising red, green and blue primary color subpixels. Themethod comprises reducing the color gamut and increasing the brightnessof the image relative to a base level, decreasing power to the displayto reduce the brightness of the image, restoring color to the image toapproximately the base level by modifying image pixel data using athree-dimensional lookup table to produce output image pixel data, andcommunicating the output image pixel data to the display to produce thecolor image. The display may be one of an LCD display, an LED display,an OLED display, a plasma display, and a DMD projector.

Reducing the color gamut and increasing the brightness of the image maybe accomplished by adding white to the image. White may be added byadding a white subpixel to each pixel of the image, or by adding whiteto at least one of the primary color subpixels of each pixel of theimage. White may be added to two or all three of the primary colorsubpixels. The white may be added to the primary color subpixels inunequal amounts.

In certain embodiments, the white may be added adaptively according toan algorithm by which the amount of white added decreases withincreasing color saturation. A Gaussian function may be used in thealgorithm to define the decrease in white with increasing colorsaturation. The algorithm may be used to determine the values in thethree-dimensional lookup table.

A plurality of images may be produced using the display, wherein thealgorithm includes determining the amount of white to add to each imagepixel and the amount of the decrease in power for each image. Thealgorithm may include determining the amount of white added for eachindividual pixel, and additionally, for each individual red, green andblue primary color subpixel. The white may be added to the subpixelsduring a portion of a pixel exposure time.

The white may be added from a second source that is separate from afirst source that provides the red, green and blue primary colorsubpixels. In embodiments wherein the display is an LCD display, a firstbacklight may be the first source, and a second backlight may be thesecond source. Alternatively, the white may be added to each pixel by awhite subpixel.

In certain embodiments, the decreasing power to the display reduces thebrightness of the image to approximately the base level. In otherembodiments, decreasing the power to the display reduces the brightnessof the image to a level brighter than the base level, i.e., some powerreduction to the display is achieved while also providing a brighterdisplay.

In certain embodiments, restoring the color of the image is performed inthe IPT color space. The values in the three-dimensional look-up tablemay be determined by using a visual model of the human visual system,which may include a model of chromatic adaptation of the human visualsystem. Restoring the color of the image may include correcting thewhite point of the display to a white point of a color standard.

In certain embodiments, memory colors are preserved in the color image.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one photograph renderedin color. Copies of this patent or patent application publication withcolor photographs will be provided by the Office upon request andpayment of the necessary fee. A Petition for the acceptance of colorphotographs is being filed under 37 C.F.R. 1.84(b)(2) concurrently withthis application.

The present disclosure will be provided with reference to the followingdrawings, in which like numerals refer to like elements, and in which:

FIGS. 1A-1D are illustrative, two-dimensional schematic diagrams ofvarious prior art ways for processing input color data to produce outputcolor data for rendering a color image;

FIG. 2 is a schematic diagram of aspects of the instant method forprocessing input color data to produce output color data for rendering acolor image;

FIG. 3 is a chromaticity diagram that depicts color gamuts of the DCIand HD709 standards, and color gamuts of various media and/or imagingdevices;

FIG. 4 is a perspective view of a three-dimensional color spacedepicting a series of color gamuts of an image display, projector, ortelevision in which the gamuts have been sequentially reduced by theaddition of white to the R, G, and B primary colors thereof;

FIG. 5 is a schematic diagram of a device for producing a color image;

FIG. 6 is a flowchart depicting the steps of one algorithm forgenerating a three-dimensional lookup table for the purposes of thisinvention; and

FIG. 7 is a flowchart depicting one method for producing a color imagein accordance with the present disclosure;

FIG. 8 is a schematic diagram of one mathematical flowchart forproducing a color image in accordance with the present invention, whichincludes color output calibration;

FIG. 9 is a graphical representation of a chromaticity diagram,including a first color gamut transformation that enables a reduction inpower consumption by a display, in accordance with the presentinvention;

FIG. 10 is a two-dimensional “slice” of the three-dimensional colorvolumes in the first color gamut transformation depicted in FIG. 9;

FIG. 11 is a graphical representation of the chromaticity diagram ofFIG. 9, including a second color gamut transformation that enables areduction in power consumption by a display, in accordance with thepresent invention;

FIG. 12 is a is a two-dimensional “slice” of the three-dimensional colorvolume resulting from a third color gamut transformation in accordancewith the present invention; and

FIG. 13 is a graphical representation of a set of color gamuttransformations in which the saturation or brightness of primary colorsis reduced, which enable a reduction in power consumption by a display,in accordance with the present invention.

FIGS. 14A-14C show image simulation results from one embodiment of theApplicants' method of displaying a color image;

FIG. 15 is a perspective view of a two-dimensional Gaussian function ofpixel saturation that may be used to calculate the pixel-dependentamount of added white to an image in an adaptive manner;

FIG. 16 depicts the dependence of the total gamut volume in CieLUV colorspace, the relative luminance increase and the power savings achievedfor various Gaussian σ values of FIG. 15;

FIG. 17 depicts a chromaticity radius within a color gamut, thechromaticity radius resulting from a chosen σ_(x) and σ_(y) value of atwo dimensional Gaussian function;

FIG. 18 depicts a comparison of the colorfulness measure with andwithout the use of one embodiment of the Applicants' color imageprocessing; and

FIGS. 19A-19D and 20A-20D depict comparisons of two exemplary imagesresulting from the Applicants' color image processing with respectiveoriginal images and unprocessed images with white added.

The present invention will be described in connection with a preferredembodiment, however, it will be understood that there is no intent tolimit the invention to the embodiment described. On the contrary, theintent is to cover all alternatives, modifications, and equivalents asmay be included within the spirit and scope of the invention as definedby the appended claims.

DETAILED DESCRIPTION

For a general understanding of the present invention, reference is madeto the drawings. In the drawings, like reference numerals have been usedthroughout to designate identical elements. In describing the presentinvention, a variety of terms are used in the description. Standardterminology is widely used in image processing, display, and projectionarts. For example, one may refer to the International LightingVocabulary, Commission Internationale de l'éclairage (CIE), 1987 fordefinitions of standard terms in the fields of color science andimaging. One may also refer to Billmeyer and Saltzman's PRINCIPLES OFCOLOR TECHNOLOGY, 3^(RD) Ed, Roy S. Berns, John Wiley & Sons, Inc.,2000; and Color Appearance Models, Mark D. Fairchild, Wiley-IS&T,Chichester, UK (2005).

In order to fully describe the invention, as used in the presentdisclosure, certain terms are defined as follows:

Brightness—attribute of a visual perception according to which an areaappears to emit, or reflect, more or less light.

BT.709—abbreviated reference to ITU Radiocommunication Sector (ITU-R)Recommendation BT.709, a standard for the format of high-definitiontelevision.

Chromaticity—normalized CIE Tristimulus values often used to visualizethe color gamuts of devices in a Chromaticity diagram, such as thatshown in FIG. 3.

CIECAM02—the most recent color model adopted by the InternationalCommission on Illumination, or Commission internationale de l'éclairage(CIE), published in 2002.

Color—A specification of a color stimulus in terms of operationallydefined values, such as three tristimulus values.

Color Space—A three-dimensional space in which each point thereincorresponds to a color.

Colorfulness—Attribute of a visual perception according to which theperceived color of an area appears to be more or less chromatic.

Contrast—In the perceptual sense, assessment of the difference inappearance of two or more parts of a field seen simultaneously orsuccessively.

DCI Standard—a color standard for digital cinema systems created byDigital Cinema Initiatives, LLC a joint venture of major motion picturestudios formed in 2002. The standard is included in the publication,“Digital Cinema System Specification,” Version 1.2 approved by DigitalCinema Initiatives, LLC Mar. 7, 2008.

Display—An imaging device which forms an image from discrete lightedelements at a surface thereof.

Color Gamut—The range of colors producible with a set of inks, lights,or other colorants. A color gamut may be described in terms of aparticular region of a color space.

Hue—Attribute of a visual perception according to which an area appearsto be similar to one of the colors, red, yellow, green, and blue, or toa combination of adjacent pairs of these colors considered in a closedring.

Memory color—a color of an object in an image for which an observer mayconsciously or unconsciously observe and make a judgment as to whetherthe color of the object is accurate, based upon the observer's memory ofprevious experiences observing the object. Examples of memory colors areflesh (human skin) tones, the green of grass, the blue of the sky, theyellow of a banana, the red of an apple, and grey scale. The accuraterendering of colors associated with commercial products and registeredtrademarks, such as “Kodak yellow”, “IBM blue,” and “John Deere green”may be important to some viewers/users of images, and are also examplesof memory colors. It is further noted that the perceived appearance ofmemory colors may be influenced by the context in which they are seen byan observer.

Primary colors—The colors of the individual light sources, including allcolor filters, that are used to create a color image in an imagerendering unit.

Projector—An imaging device which forms an image by delivering and insome instances focusing light on a distant, separate surface such as awall or screen.

RGBCYMW—in the use of any of these capital letters in combinationherein, they stand for red, green, blue, cyan, yellow, magenta, andwhite, respectively.

Rendering an image—providing an image for observation, either via animage display that forms an image from discrete lighted elements at asurface thereof, or via an image projector that forms an image bydelivering and in some instances focusing light on a distant, separatesurface such as a wall or screen.

Saturation—Colorfulness of an area judged in proportion to itsbrightness.

Secondary colors—Linear or non-linear combinations of the primary colorsof an image rendering unit that can be controlled independently from theprimary colors.

Tristimulus values—Amounts of the three reference color stimuli, in agiven trichromatic system, required to match the color of a stimulusbeing considered.

White—a set of three values of primary colors, typically red, green, andblue, that may be added to a color in a portion of an image, thereby ineffect adding white to the color to brighten the color.

It is further noted that as used herein, a reference to a threedimensional lookup table or a 3DLUT is meant to indicate a table of atleast three dimensions, unless otherwise indicated. A lookup table maybe multidimensional, i.e., it may have three or more input colors andthree or more output colors.

FIG. 2 is an illustrative, two-dimensional schematic diagram depictingthe full multi-dimensional capability of an at least three dimensionalcolor table 54 used in processing input color data to produce outputcolor rendering a color image. For the sake of simplicity ofillustration, the diagram 420 of FIG. 2 depicts only a 2D rendition ofan at least 3D color table 54 of the present invention. Any point,and/or any region in the full color space can be changed independently.The small squares 422 represent locations in the color space in which nochange in color is made. These locations may be memory color locations,such as flesh tones.

In other regions 424, selective increases in contrast, colorfulness, andbrightness may be made. The larger squares 426 in these regions 424represent locations where colorfulness, contrast, and brightness areincreased. Any local color or color region, such as a flesh tone region,can be chosen for unique color processing. In one embodiment, a 3D colortable may contain output values for every input RGB color, which for 12bits per color would be 4096×4096×4096 independent colors, therebyproviding 68.7 billion local color choices. In another embodiment, a 3Dcolor table size can be reduced by using the most-significant bits ofthe input colors to define the 3D color table locations and performingmulti-linear or other multi-dimensional interpolation using theleast-significant bits of the input colors.

It is to be understood that the while the squares 422 and 426 are meantto indicate various color regions, the borders of the squares are notmeant to indicate sharply defined boundaries of such regions. Asdescribed previously, these regions may be modeled using a probabilitydistribution that provides a smooth transition from regions in the colorspace that are outside of the regions defined by the squares.

For example, the various regions may be defined by Gaussian boundariesthat are smoothly connected by probability functions. In defining thecolor output values in the at least 3D LUT 54, volume derivatives may beused that displace the color (R,G,B) vectors in different amounts.Within memory color regions, the color vectors have a lesserdisplacement, or possibly none at all, while other color regions havelarger displacements to increase their contrast, colorfulness, andbrightness.

The full table may be very large. For example, a large table results ifthe input color is 24-bit (i.e. 8 bits each for R, G, and B), and theoutput includes white and is 32 bit (i.e. 8 bits each for R, G, B, andW). Referring to FIG. 5, this large 3D LUT 54 may be used if the memory36 of the image color rendering controller is sufficiently large, andresults in the fastest color processing. However, if the memory 36 islimited in size, but sufficient computational capacity is available inthe CPU 34, multi-dimensional interpolation may be used to reduce thesize of the 3D LUT 54. In this particular example, for each respectiveprimary input color, bits 3 through 8 may be used to define and addressthe 3D LUT 54. Multi-dimensional interpolation may then be used withbits 1 and 2 to define the output colors that occur between the outputcolors associated with the 8 vertices of the cube in the 3D LUT 54defined by bits 3 through 8.

The color gamut of an image rendering unit, such as a display,television, and/or projector is defined by the maximum colors that canbe produced by that image rendering unit with combinations of itsprimary colors. FIG. 3 shows the color gamuts of various image renderingtechnologies compared to the CCIR709 color standard 404 and the DCIcolor standard 402. FIG. 3 shows that displays such as LED projectors(gamut 406), OLED displays (gamut 408), Digital Cinema projectors (gamut410) and televisions with more than 3 primary colors (gamut 412) havelarger color gamuts than the CCIR709 color standard (gamut 404) fordigital media distribution, thus illustrating the need to map thesmaller CCIR709 color standard to the larger color gamut of thesedisplay types. All other international color standards for consumerdigital color media are similar to CCIR709 and therefore exhibit thesame need to map these standards to the larger color gamut of thedisplay types in FIG. 3. In the methods of the present invention, thisis done while simultaneously preserving memory colors, and optimizingthe particular device for viewing in a particular environment, andtaking into account adaptation of the human visual system. FIG. 3 alsoshows that the DCI “Hollywood” color standard is significantly largerthan the color gamut 414 of an infinite set of lasers, and thereforelarger than any possible display or image rendering unit, thusillustrating the need to map the larger input to the smaller color gamutof any display type including a professional digital cinema projector.

In a color image rendering unit, such as a display, television, and/orprojector, in order to achieve maximum brightness with a single opticalsystem and single image modulator, the multiple RGB channels may becombined for some portion of time during image frames. Adding thesemultiple RGB channels during an image frame duty cycle will increase thebrightness of the image, but will also reduce the colorfulness bydesaturating the pure RGB colors. FIG. 4 is a perspective view of athree-dimensional CIECAM02J L*a*b* opponent color space 10 depicting aseries of color gamuts of an image display, projector, or television inwhich the gamuts have been sequentially reduced by the addition of whiteto the R, G, and B primary colors thereof. The outer (coarsest squares)color gamut 12 is the color gamut of one exemplary image projectorhaving its primary colors produced by red, green, and blue LEDs. Thewire frame color gamut 11 represents the CCIR709 video color standard.The successively finer squares solids 14, 16, 18, and 20 represent thecolor gamuts resulting from the addition of 6.25%, 12.5%, 25%, and 50%white, respectively. For the sake of simplicity of illustration, 2Dprojections of the color gamuts 11-20 are provided on the a*b* plane asrespective closed curves 11A-20A. The color gamut 12/12A of the LEDprimaries has no added white. It can be seen in general from the 3Dperspective renditions and the 2D projections that the addition of whitealways reduces the color gamut of the image device.

However, this does not mean that the addition of white to the images ofthe device cannot be beneficial. It can also be seen that the additionof white at a 6.25% level, as indicated by solid 14 and closed curve14A, results in a color gamut that is approximately equal to the CCIR709color video standard, while at the same time making the image perceivedto be brighter. In an image rendering unit, and particularly in singlemodulation LED displays such as those employing a digital micromirrordevice (DMD), the image is made to appear brighter by the addition ofwhite from combining RGB colors. In digital cinema, this may be done forsome portion of the image frame time. The capabilities of human visualsystem adaptation are thereby exploited to increase the apparentbrightness and lightness contrast of the displayed images.

In one aspect of the present invention, visual models of visualperception by the human visual system are used in determining theoptimum amount of white to add to the colors of the image. The perceivedcolorfulness, contrast, and/or brightness of the image are enhanced,thereby improving the perceived quality of the image. The visual modelsof human visual perception may be used to create look-up tables of atleast three dimensions to process the image to be displayed. The methodsof the present invention may include performing empirical visual studiesto determine the dependence of preference of colorfulness, contrast, orbrightness on the ethnicities of the human observers, and defining theperceived quality of the image for each nationality of human observers.The colorfulness, contrast, or brightness of the image may be adjustedbased upon the preferences of one of the ethnicities of the humanobservers.

FIG. 5 is a schematic diagram of a device for producing a color image,which may be observed by a human observer. The imaging device mayinclude an image rendering unit such as e.g., a television, a display, aprojector, or another unit. Referring to FIG. 5, the imaging device 30may include an image color rendering controller 32 or computer 32 orother processor comprising a central processing unit 34 and a memory 36.As an alternative memory, or in addition to the memory 36, thecontroller 32 may include a computer readable storage medium 38 such asa hard disk. These components are in communication through a system bus39. The device 39 may be further comprised of an image rendering unit40, which may be an image display or projector, such as a liquid crystaldisplay 42; a plasma display 44; a digital mirror device (DMD) 46including a DMD 80, a lamp 82, and color wheel 84; or a digital mirrordevice 48 including a DMD 80, and red, green, and blue LED's, OLEDs orlasers 86, 87, and 88.

The imaging device 30 may process input image data that is stored on thestorage medium 38, or the imaging device 30 may receive input image datafrom an external device or source 50. The external source 50 maycomprise an Internet connection or other network or telecommunicationsconnection, such that the input image data is transmitted through suchconnection.

The imaging device 30 may be adapted to a system for displaying orprojecting an image in a variety of ways, depending upon the particularapplication. In some embodiments, the imaging device 30 may be providedas an integrated system comprising the controller 32 and the imagerendering unit (display or projector) 40, which only needs to beconnected to a source 50 of image input data. In another embodiment, theimaging device 30 may be separate from the image rendering unit 40, andin communication with the image rendering unit 40 through a network ortelecommunications connection as described above. The imaging device 30may be provided comprising the image color rendering controller 32, afirst port (not shown) for connection to a source 50 of image inputdata, and a second port (not shown) for connection to the imagerendering unit 40. This configuration is particularly useful forretrofitting to projection or flat screen televisions that receivesignals via a cable that is connected to a broadcast source of imageinput data (e.g., “cable TV programming”). In such circumstances, thecable carrying input image data 50 could be disconnected from the imagerendering unit 40, and the imaging device 30 could be placed in linebetween them to perform the image processing of the present invention.

In other embodiments, the imaging device 30 may be in communicationwith, or integrated into an auxiliary device 60 or auxiliary imagingdevice controller 60, which is in communication with the image renderingunit 40. The imaging device controller 60 may be, without limitation, anaudio/video processor, a cable TV set-top box, a video game console, apersonal computer (PC), a computer graphics card of a PC, or a DVD orBlu-ray player. In another embodiment, the imaging device 30 may beintegrated into the electronics and processing components of a broadcaststation, a broadcast antenna, receiver or processor, or a digital cinematheatre. In another embodiment, the device 30 may be integrated into thehardware and software of media creation, preparation, and productionequipment, such as equipment used in the production of DVDs of moviesand television programs, or the production of digital cinema fordistribution to theaters. Broadcast stations, digital cinema theaters,and media production equipment may all be comprised of an auxiliaryimaging device controller 60.

The memory 36 of the device 30 may contain a set of at least threedimensional lookup tables 54; each table of the set may be optimized fora different viewing environment of the image rendering unit 40. Thedevice 30 may be provided with a sensor 70 for measuring the ambientlight in the viewing environment of the image rendering unit 40, or inthe case of a projector 46 or 48, in the viewing environment of theprojected image. The memory 36 may contain a visual model of theperception of the human visual system that may be used to enhance theperceived colorfulness, contrast, or brightness of the produced image.

FIG. 6 is a flowchart depicting an algorithm for generating athree-dimensional lookup table to improve the perceived colorfulness,contrast or brightness in non-memory colors, while preserving to ahigher degree the color accuracy of memory colors. The algorithm 100 ofFIG. 6 may be used to perform step 210 of the method 200 of FIG. 7.Additionally, the algorithm 100 is applicable to other image renderingdevices that use DMD, plasma, liquid crystal, liquid crystal-on-siliconmodulation, or direct modulation of the light source, and using LED,OLED, laser, or lamp light sources.

Referring to FIG. 6, in operation 110, the RGB input values of the inputimage data are “reverse gamma” corrected to compensate for thenon-linearity of this data, thereby producing linearized scalar RGBvalues. (The original input data is supplied with the expectation thatit will be used in a display or projector that may have a gamma value ofabout 2.2, for example.) In operation 120, the outer product of thescalar RGB values and the projector matrix is taken to express the inputimage data as CIE XYZ tristimulus values. In operation 130, thetristimulus values are converted to a visual color space. Thetransformation to a visual color space enables perceptual modeling to beperformed, which characterizes the interdependencies of color, contrast,and brightness, and allows the perception of memory colors to bepreserved. The visual color space may be an opponent color space thataccurately models constant perceived hue, and has the dimensions oflightness, yellow-blue, and red-green.

In operation 140, the visual color space predicted appearance attributesof lightness, chroma, and hue are computed. In operation 150, theenhanced lightness, chroma, and hue for colors to be rendered arecomputed. Operation 150 may include steps 152, 154, and 156 formaintaining memory colors in the rendering of the image.

In applications in which there are specific memory colors to bepreserved, operation 150 of the method 100 may include steps 152, 154,and 156. More specifically, the method 100 may include the step 152 ofidentifying the memory colors in the input image data 50 to besubstantially maintained. This may be done based on intuition andexperience and/or market research data. It is known that observers of animage depicting human subject matter (such as a movie or televisionprogram) will find it objectionable if the colors of the skin, and facesin particular, of the humans in the image do not match those colors thatthey have in their respective memories of how the humans should look.They will perceive the humans as “not looking right,” if they are toopink, orange, dark, light, etc. In like manner, certain other memorycolors, such as “grass green” and “sky blue” must be rendered so as toappear as the observers remember them from experience. Regardless of howsatisfactory the other colors in the image appear, the observers willfind a product that does not render memory colors accurately to not beperceptually optimal, and will likely not buy the product, whether theproduct is an imaging device such as a television, or a movie to beviewed in a theater.

Once the memory colors are chosen, they are characterized with respectto their chromaticities in step 154 from both empirical data and theperceptual context in which they are seen. For instance, it is wellunderstood that humans remember green grass and blue sky as moresaturated than the actual stimuli. And, within reason, no matter thecolor of an illuminant, humans will remember a banana to appear to be acertain yellow (which may also be a memory color). Furthermore, thesememory colors are not distributed across the extent of perceptual colorin any systematic way. Hence, their representations must necessarily bemade in a multivariant, three dimensional, statistical sense and theirrendering accomplished in a purely appearance or vision based colorspace. Algorithms may be employed using visual mathematics which ensurethat the memory colors are specified in terms of perceived colors.

In step 156, the enhanced lightness, chroma, and hue for non-memorycolors and memory colors are also computed. It is noted that in thecolor space of the input image data, a given memory color is not asingle point within the space. To the contrary, memory colors areregions within the color space that are to be left at least perceptuallyunchanged, or much less changed during the color transformations of theinstant methods to produce enhanced images. By way of example, thememory color “flesh tone” is a range of colors corresponding to thecolors of very dark-skinned peoples of African ethnicity to very lightskinned Caucasians or Asians. Accordingly, the memory colors areidentified and characterized such that the colors within this regionwill be left unchanged or minimally changed in the colortransformations.

Additionally, these memory colors may be characterized as not havingrigid, discrete boundaries; this may be done so that in the colortransformations to be performed, there is not a discontinuity in thedegree of color change at a boundary of a memory color, as explainedpreviously with reference to FIG. 2. In one embodiment, the memory colormay be modeled using a probability distribution that provides a smoothtransition from regions in the color space that are non-memory colors tothe region defined as the particular memory color. Any smoothingfunction that changes the local multi-dimensional derivatives smoothlywill be satisfactory. The probability distribution may use non-linearenhancement functions. An exemplary overall non-linear function that maybe used is

${Output} = {0.0001 + \left( \frac{1.5 \times {Input}^{EXP}}{0.5 + {Input}^{EXP}} \right)}$

In operation 160, the enhanced lightness, chroma, and hue of the visualcolor space are converted to enhanced CIE XYZ tristimulus values. Inoperation 170, the enhanced CIE XYZ tristimulus values are converted toenhanced RGB scalar values with “white channel.” In operation 180, gammacorrection of the enhanced RGB scalar values is performed to produce a3DLUT containing enhanced RGB values with white channel. The 3DLUT maythen be used in the method 200 of FIG. 7.

FIG. 6 concludes with a simple statement 101 of the net effect of theoperations 110-180. The 3DLUT, which is of at least three dimensions, iscreated as a discrete sampling of the visual model andcontrast/color/brightness HVS perceptual improvement mathematics, andmay include preservation of memory colors. Referring also to FIG. 5, theat least 3DLUT 54 may be generated by the CPU 34 of the imaging system30 according to an algorithm 52 stored in memory 36 or on the readablestorage medium 38. Alternatively, the at least 3DLUT 54 may be generatedby another computing system and uploaded to the system computer 32. Thealgorithm 52 of FIG. 5 for generating the at least 3DLUT 54 may bealgorithm 100 of FIG. 6.

FIG. 7 is a flowchart depicting one method for rendering a color imagein accordance with the present disclosure. The method may be performedusing the imaging system 30 depicted in FIG. 5. Referring again to FIGS.5 and 7, in step 210, the 3DLUT 54, which may be produced according tothe algorithm 100 of FIG. 6, is loaded into the memory 36 or thereadable storage medium 38 of the imaging device 30. In step 220, theinput image data from the source 50 is communicated to the CPU 34. Theinput image data may be of a first input color standard, and may beconverted into an input color specification for inputting into the atleast three-dimensional look-up table. In step 230, the input image datais processed with an algorithm 56 that may be stored in memory 36, usingthe at least three-dimensional look-up table 54 to produce renderedimage data. In step 240, the rendered image data is output to the imagedisplay/projection device 40, and a high brightness, high contrast, andhigh colorfulness image is displayed or projected in step 250. The imagemay include human visual system perceptually accurate memory colors. Themethod 100 may be repeatedly performed at a high rate on sequences ofimage input data, such as at the rate of 24 or 48 “frames per second”used in digital cinema, or such as at the rate of 30, 60, 120 or 240frames per second used in consumer displays.

Referring again to FIG. 5, and in one embodiment, the 3DLUT 54 of inputcolors and output colors may contain, or the values therein may bedetermined from, the definition of secondary colors, and enhancedlightness, chroma, and hues to increase perceived colorfulness,contrast, or brightness to compensate for the loss in perceivedcolorfulness, contrast, or brightness due to addition of secondarycolors by the image rendering unit 40. In another embodiment, the 3DLUT54 of input colors and output colors may contain, or be determined from,a transformation from a suboptimal viewing environment to an improvedviewing environment including the visual adaptation of the human visualsystem.

In another embodiment, the method may include providing input image data50 of a first color gamut, and an image rendering unit 40 having asecond, expanded or reduced color gamut. The 3DLUT 54 of values of inputcolors and output colors is generated, wherein the values in the 3DLUT54 are calculated based upon a visual model of the human visual system,thereby expanding the input image data 50 to encompass the second colorgamut of the image rendering unit 40.

In another aspect of the invention, the image rendering unit 40 may beprovided with some color modification capability that is built in orembedded in hardware or software. For example, the device may beprovided with an algorithm to add white or secondary colors, resultingin a loss of colorfulness, and a distortion in the appearance of memorycolors. In such circumstances, the output values in the 3DLUT 54 aredetermined such that the input image data 50 is processed to compensatefor the color modification performed by the image rendering unit 40. Themethod may thus include providing the 3DLUT 54 to adjust the color datain a manner that shifts it in a direction within the color space thatcompensates for the embedded color modification that is performed by theimage rendering unit 40. The 3DLUT 54 further comprises processing theinput image data to increase perceived color, brightness, and contrastto compensate for the reduction in perceived color, brightness, andcontrast caused by the algorithm for color modification in the imagerendering unit 40.

In a more specific instance in which the image rendering unit 40includes an algorithm for creating secondary colors from primary colors,the 3DLUT 54 may further comprise compensating for the colormodification performed by the addition of the secondary colors in theimage rendering unit 40. The values in the 3DLUT 54 may also bedetermined such that the 3DLUT 54 further comprises processing the inputimage data 50 to include chromatic adaptation of the human visual systemto a specified white point that increases the brightness of the imagerendering unit 40.

In another aspect of the invention, the image rendering unit 40 mayunintentionally contain some color modification capability resultingfrom variation in one or more parameters of the unit 40. For example, ifthe image rendering unit 40 is an OLED display, then over the life ofthe display, color modification may occur due to the differing lifespans between blue OLED and red and green OLEDs of the display, asdescribed previously herein. During the operation of the OLED display,the differential color change between the blue OLED and the red andgreen OLEDs will change the color balance of the display if nocountermeasures are instituted.

In such circumstances, the output values in the 3DLUT 54 may bedetermined such that the input image data 50 is processed to compensatefor the predicted decrease in luminance of the blue OLED. The method maythus include providing the 3DLUT 54 to adjust the color data in a mannerthat shifts it in a direction within the color space that compensatesfor decreasing blue OLED luminance. The 3DLUT 54 further comprisesprocessing the input image data to increase perceived color, brightness,and contrast to compensate for the reduction in perceived color,brightness, and contrast caused by the continual loss of blue OLEDluminance.

The 3DLUT 54 may also adjust the colorfulness, contrast, or brightnessof the image to be produced to appear as it would in an image from ananalog film system or digital system used in cinemas. It is known thatfilm is generally not designed to reproduce color as the eye sees it atthe filming site. (A color gamut 416 for film is shown in FIG. 3.)Instead, the colors in film images have increased contrast and increasedcolorfulness in anticipation of the viewing environment in which thefilm images will be observed. It is also known that digital systems aimto match the look of film images. Accordingly, the 3DLUT may be designedto provide the same effect in a cinema.

The production of the 3D LUT 54 is not limited only to the algorithm 100of FIG. 6. Bit depth modification and interpolation as described hereinmay also be applied to all of the applications herein which include theuse of 3DLUTs. The 3DLUT may vary in bit depth, depending upon thecapacity of the memory 32 and the processing power of the CPU 34. In oneembodiment, the 3DLUT may be a twelve bit table with 4096×4096×4096discrete addresses containing three or more color values ofpredetermined bit precision. In another embodiment, some bits of thetable may be used for interpolation between adjacent values. Forexample, the final two bits of respective adjacent table values may beused in interpolating colors between them. Other methods ofmulti-dimensional interpolation are known, and are included inembodiments of implementing the 3DLUT. Additionally, the input data maycontain more than three primary colors such as RGB. For example, theinput data may contain RGBCMY (wherein C=cyan, M=magenta, and Y=yellow),or some lesser combination such as RGBCM. In such an instance, the 3DLUTcould have outputs of RGBCMYW.

Depending upon the particular application, the algorithm 100, or otheralgorithms that may further include bit depth modification andinterpolation, may be used to produce more than one 3DLUT. One factorthat may be used to determine the values in the 3DLUT is the set ofcharacteristics of the display or projection device. Referring again toFIG. 5, different 3DLUTs 54 may be produced for different image outputdevices, for example, an LCD display 42, a lamp-and-color-wheel DMDprojector 44, and an LED DMD projector 46. The characteristics of thedisplay or projection device 40 include the “color engine” of thedevice, and whether it includes only RGB as the primary colors, or hasmore than three colors. The 3DLUTs 54 may be losslessly compressed toreduce storage use in the memory 36 of the image color renderingcontroller 30.

Other factors pertain to the “surround,” i.e., the viewing environmentof the display or projection device 40, such as the ambient lighting ofthe room in which the display or projection occurs, and the lightingand/or surface immediately surrounding the display/projection screen. Ingeneral, the 3DLUT values provide a displayed/projected image havingmore contrast, brightness, and colorfulness for any “surround”, i.e.viewing environment; for example, a particular room lighting and anyconversion from that room lighting to an improved room lighting. If theroom lighting is darker or brighter than a desired level, the generationof the 3DLUT 54 may include a visual adaptation transformation toproduce a perception of improved viewing environment. The visualadaptation transformation is based upon visual models that may includemodels of the adaptation of the human vision to viewing environments.

For example, in a dark room there is essentially no ambient lighting(other than minimal safety and exit lighting), but using a visualadaptation transformation to increase contrast and colorfulness in amanner analogous to that used in motion picture print film to providethe perception of an improved viewing environment to an observer. As theroom lighting increases and the image brightness increases to about thesame level, the adaptation transformation is still needed because it theroom lighting is still not as bright as daytime outdoor lighting, whilethe ambient lighting must be compensated for. In summary, the visualadaptation transformation implemented in the 3DLUT 54 uses visualadaptation models to produce the effect of improved viewing environment.

Other factors in generating the 3DLUT 54 may include a knowledge of thedifferent sensitivities to colorfulness in different worldwide regions,or the intended use of the displayed/projected images; for example,whether the images are viewed in a video game that is being played, orviewed as a movie or television program.

These multiple 3DLUTs 54, or a subset of them may be stored in thememory 36 of the computer 32 of the device 30. Additionally, data on theviewing environment factors 58 may be stored in memory. The image device30 may include a keyboard (not shown) or other input device to access auser interface (not shown) that may be displayed on the display orprojector 40 (or other user interface screen). The user interface mayoffer the capability of inputting data on the viewing environmentfactors 58, and/or other factors such that the optimum 3DLUT is selectedfrom the stored 3DLUTs 54 for the particular display or projector 40 andviewing environment. In that manner, the most perceptually optimalimages are provided to the user by the system 30. The 3DLUTs 54 areeffective for the enhancement of a variety of images, including but notlimited to games, movies, or personal photos. Additionally, someimprovement of grey scale images is attained by the resulting contrastand brightness enhancement thereof.

The 3DLUT 54 may be produced according to variants of the method 200such that it has additional or alternative characteristics. The valuesin the 3DLUT 54 may be provided to convert a first color gamut of aninput image data set 50 to encompass a second expanded or reduced colorgamut of an image rendering unit 40 that is connectable to the device30. The 3DLUT 54 may contain a transformation from a suboptimal viewingenvironment to an improved viewing environment in which the color imageis to be observed, including the visual and chromatic adaptation of thehuman visual system. The 3DLUT 54 may contain the definition ofsecondary colors, and enhanced lightness, chroma, and hues to increaseperceived colorfulness, contrast, or brightness to compensate for theloss in perceived colorfulness, contrast, or brightness due to additionof secondary colors by an image rendering unit 40 that is connectable tothe device 30.

In another aspect of the invention, the methods of producing a colorimage may include input color standard transformation and color outputcalibration of the image rendering device that is in use. This is bestunderstood with reference to FIG. 8, which is a schematic diagram of analternative method 300 for producing a color image, which includes suchcolor output calibration. The diagram includes color output calibrationoperations 350, 360, and 370; however, for the sake of clarity, theentire method depicted in FIG. 8 will be described, with reference alsoto FIGS. 6 and 7.

In operation 310 (“Gamma1”), the input values of R, G, and B are reversegamma corrected to compensate for the non-linearity of this input datastandard, thereby producing linearized scalar values R_(i), B_(i), andG_(i). This correction may be done using the respective one dimensionallookup tables 311, 312, and 313. The input values of R, G, and B may bebetween 8 and 12 bits (314 in FIG. 8) inclusive. The output values ofR_(i), G_(i), and B_(i) may have 16 bit resolution (315 in FIG. 8),depending upon the architecture of the image color rendering controller32, and also upon the need for the greater bit depth of the imagingstandards being used. The input R, G, and B values may be provided fromvarious devices, such as a video camera having an output in accordancewith standard BT.709. In such circumstances, the value of gamma used inthe correction may be 2.2. The input R, G, and B values may be providedin accordance with other imaging standards, and other values of gammaand other 1D lookup tables 311, 312, and 313 may consequently be used inthe reverse gamma correction as needed.

In operation 320 (“Color Transform”), every color value in the imagedata stream 319 represented by a unique R_(i), G_(i), and B_(i)combination is then operated on by a 3×3 matrix determined by theparticular imaging standard being used to perform a color transformationto R_(ii), G_(ii), and B_(ii) values that are linearized scalar valuesreferenced to the standard BT.709. The R_(ii), G_(ii), and B_(ii) valuesmay be provided with a bit resolution of up to 16 bits as indicated inFIG. 8.

In operation 330, (“Gamma2”), the values of R_(ii), G_(ii), and B_(ii)are gamma encoded to re-introduce a non-linearity into the processeddata, thereby producing gamma encoded values R_(iii), B_(iii), andG_(iii) for input to the 3D color tables. This encoding may be doneusing the respective one dimensional lookup tables 331, 332, and 333,using a gamma encoding factor of 1/2.2, in one embodiment. Other factorsmay be suitable, depending upon the particular imaging standards beingused. The resulting values of R_(iii), B_(iii), and G_(iii) may bereduced to 10 bit resolution as shown in FIG. 8, to enable sufficientlyfast subsequent processing using the 3D color tables 54. The gammaencoding enables a reduction in the number of bits from 16 for lineardata to much less for gamma encoded data, such as 10 bits, withoutartifacts. This makes the at least 3D table much smaller. It iseffective to use fewer gamma encoded bits because the eye sees imagedata in a manner analogous to a gamma encoder.

In operation 340, the three dimensional color tables 54 are used toprocess the R_(iii)B_(iii)G_(iii) data to produce output imageR_(iv)B_(iv)G_(iv)W_(iv) data for display or projection. In thisembodiment, the table 54 is 3D in (RGB) and 4D out (RGBW). Other tablestructures of at least three dimensions may be used, depending upon theparticular application. Additionally, for the sake of simplicity ofillustration, there is only one table 54 shown in FIG. 8; however, it isto be understood that there is a first 3D LUT for determining R_(iv), asecond 3D LUT for determining G_(iv), a third 3D LUT for determiningB_(iv) and a fourth 3D LUT for determining W_(iv), where a white channelis implemented. In this embodiment, the white could be for an OLEDdisplay, or the signal that drives the combination of RGB to make theimage rendering device brighter. Alternatively, the white could bereplaced with cyan, or some other color in a four-color image renderingdevice, such as a four-color TV. The R_(iv)B_(iv)G_(iv)W_(iv) data maybe provided at a 12 bit resolution as indicated in FIG. 8.

At this point, the R_(iv)B_(iv)G_(iv)W_(iv) data, including the additionof white for increased brightness or color management of OLED devicesmay represent a generic display with typical color primaries andlinearity. Additionally, however, further operations may be performed tofurther optimize the R_(iv)B_(iv)G_(iv)W_(iv) data by calibration forthe particular image rendering unit (display or projector) 40 that is inuse. The measurement or specification of this particular image renderingunit 40 can be done in manufacturing on done on-site by a technicianwith conventional linearity and primary color measuring tools.

Referring again to FIG. 8, in operation 350 (“Gamma3”), theR_(iv)B_(iv)G_(iv)W_(iv) data is first reverse gamma-corrected toproduce R_(v)B_(v)G_(v)W_(v) data. This correction may be done using therespective one dimensional lookup tables 351, 352, 353, and 354. Theoutput values of R_(v), G_(v), B_(v), and W_(v) may have 16 bits. Thevalue of gamma used in the correction may be 2.2, or another value inaccordance with the gamma encoder 310.

In operation 360 (“Color Calibration”), every color value in the imagedata stream 359 represented by a unique R_(v), G_(v), B_(v), and, and inmany cases, W_(v) combination is then operated on by a 4×4 matrix. This4×4 matrix is produced for and is unique to the particular imagerendering unit 40 of FIG. 5 that is in service. The matrix is calculatedfrom measured or specified values that define the color primaries of theparticular image rendering unit 40. The purpose of the operation is toconvert from the assumed or generic color primaries in the at least 3Dcolor table to the actual ones in the image rendering unit 40. Thevisual effect is to adjust for white and the rest of the colors so theyare not “tinted” (e.g., a little yellow or blue), because the imagerendering unit may have slightly different color primaries than wereassumed in creating the at least 3D table. For standard televisions orprojectors, those assumptions are in accordance with the aforementionedBT.709 standard, because most TVs, displays, and projectors adhere tothis standard. A given image rendering device may be tinted, e.g., moreyellow, however so the calibration matrix compensates for thatvariation. The R_(vi), G_(vi), B_(vi), and W_(vi) values may be providedwith a bit resolution of up to 16 bits.

In operation 370, (“Calibration”), the R_(vi), G_(vi), B_(vi), andW_(vi) values are gamma encoded to introduce the correct non-linearityinto the processed data for the image rendering unit 40, therebyproducing the R_(vii), G_(vii), B_(vii), W_(vii) values that, when usedby the particular image rendering unit 40 to project or display theimage, produce chosen non-linearity defined by the 3D table. Thisencoding may be done using the respective one dimensional lookup tables371, 372, 373, and 374. In one embodiment, a gamma encoding factor of1/2.2 may be used. Other factors may be suitable, depending upon theparticular imaging rendering unit 40. The resulting values of R_(vii),G_(vii), B_(vii), W_(vii) may be output having between 8 and 12 bitresolution as indicated in FIG. 8.

In another aspect of the invention, the problem of displaying highquality images on a portable display device over an extended period oftime is solved by modifying the primary colors of the display devicesuch that the resulting new primary colors are more efficient. Thisenables power to the device to be reduced, such as by using a lowerpower light source (for a liquid crystal display), or by using a lowerpower lamp or lower power LEDs or OLEDs of primary colors or white. Thisresults in less heat production and less other display management costs.

In certain embodiments, adaptive color processing is used to improveimage quality in any ambient lighting. The red, green, and blue primarycolors of the display are redesigned to provide increased efficiency andbrightness. This enables reducing power consumed by the display back tothe initial brightness levels of the unmodified display.

It is known that when the brightness of a display is reduced in dark anddim lighting, the color is also reduced. To the best of the Applicants'knowledge, heretofore there have been no satisfactory methods to addressthat problem. The redesigns of color primaries as referenced above areaccompanied by significant losses in color saturation. These losses arerecovered through the use of three dimensional lookup tables (3DLUTs).The Applicants 3DLUTs are determined using visual models as describedpreviously herein, which provide completely independent output colordesign capability to compensate for color losses in different ambientlighting. These models of the human visual system include chromaticvisual adaptation. The application of the Applicants' methods ofdefining and using 3DLUTs to power savings in displays will now beexplained.

In the instant methods, color losses are compensated for when displaybrightness is reduced; and for smaller gamut displays, color losses arecompensated for when color primaries are adjusted to be more efficient,i.e., brighter, for power savings. There are a variety of ways to modifythe sources of primary colors of a display to make it more efficient andto increase brightness. In an LCD display comprising a backlight,because power and brightness are directly related, the brightnessincreases can be used directly to reduce power. For example, a 100%increase in brightness can be used to decrease power 50%, according tothe equationPR=1−1/Bwhere PR is the power reduction and B is the decimal brightness. It canbe seen that for a 100% brightness increase, i.e., a doubling of thebrightness, B=2.0 and PR=0.5=50%.

Although a number of methods to modify the color primaries have beensuggested by others for mobile devices, they have all been difficult toimplement because they cause a significant loss in color saturation andcolor hue errors while lacking methods to restore the colorfulness ofthe display.

In contrast, the methods disclosed herein restore the colorfulness formodified chromaticity color gamuts and desaturated color primaries. Themethods of color mapping use 3DLUTs, which are determined using visualmodels and colorfulness increases based on visual compensation forambient lighting losses in colorfulness. Using visual models andcompensation for colorfulness loss in various ambient lighting is apreferred approach to restore the colorfulness with color primarychanges, because the color increases are very natural perceptually,since they represent what colorfulness a human observer would perceivein better lighting conditions. The use of visual models enables smoothincreases in colorfulness throughout the full color gamut volume at allbrightness levels.

In building a 3DLUT, a gamut mapping method is also used, which avoidsloss of detail due to brightness and hue changes that are common in morestandard gamut mapping approaches. This is beneficial because there aresignificant color gamut issues with modified chromaticities. (See forexample, Daly, et al., “Gamut Mapping in LCD backlight compensation”,16^(th) Color Imaging Conference, May 31, 2011.)

In certain embodiments of the instant method, significant brightnessincreases and power savings are achieved by adding an adaptive andcontrolled amount of white in a 4^(th) sub pixel to an image display ofa standard backlight LCD display; while restoring the correspondinglosses in colorfulness by the use of the Applicants' 3DLUTs. Because theadded white and increased brightness is adaptive and different for everypixel in a given image, the brightness increase and resultant powersavings are image dependent. As will be explained, more power savingsare available for images that are of lower color saturation and/or moreblack and white with less color. A brightness increase map for an imagewill be provided, with the average power savings equal to (1−1/B_(ave)),where B_(ave) is the average new brightness increase. Accordingly, a100% average brightness increase has a B_(ave)=2.0 and a power savingsof 50%, as noted previously. In practice, the desired power savings maybe set to an amount selected through analysis of a typical set ofimages. For any given image, if the selected power savings to beattained is too high, the image will be slightly dimmer, and if it istoo low the image will be slight brighter. To compensate for a slightlydimmer image, the power savings may be biased lower that the optimumavailable amount to provide some extra brightness in the overallprocessing and the resulting images.

As an alternative method, the Applicants have considered adding staticamounts of white to every pixel of an image, thereby decreasing thecolor saturation for all levels of pixel saturation. This requires themost color restoration because the most saturated pixels lose asignificant amount of color and the color gamut with added white isreduced significantly. In analyzing this approach, it was discoveredthat a better approach is to add white and brightness adaptively withmore white added to less saturated pixels. This allows more white andbrightness increase on average with higher overall power savings andless loss of color saturation and gamut prior to the use of theApplicants' 3DLUTs to produce the output images.

Results from one example of adding static amounts of white to everypixel of an image indicated a brightness increase of 105%, whichcorresponded to a power savings of about 51%. In contrast, using theApplicants' 3DLUTs resulted in the calculated colorfulness from the lossin color gamut (compared to the sRGB color gamut) due to adding whiteincreased from 22.6 to 43.3 or an increase of 91%. These results aresummarized in TABLE 1.

TABLE 1 Comparison of brightness, power savings and colorfulness forsRGB and static white sub pixel amounts using optimal block dyes andchromatic adaptation to the new white point with differing amounts ofadded white for each primary. Colorfulness Colorfulness Measure withoutMeasure with % use of 3DLUTs use of 3DLUTs Color U % Open LightnessBrightness Power and chromatic and chromatic Primaries White (lux)Increase Savings adaptation adaptation sRGB 14.6  0 0  22.6 Optimal 10%Blue 29.9 105% 51% 23 43.3 Block Dyes 20% Red 30% Green

As will be explained presently, this colorfulness measure is linear withthe colorfulness ranking by users for a large sample set of images.Power savings may be achieved by adding white (or “open filter” regionsin LCD displays) to each pixel with high color saturation using theApplicants' 3DLUTs and chromatic white point adaptation. FIGS. 14A-14Cshow image processing results from the instant method that is summarizedin TABLE 1. FIG. 14A is an original sRGB image that may be displayed byan imaging device at a given level of power consumption. FIG. 14B is theimage that results from adding 10% white to blue, 20% white to red and30% white to green for each pixel of the image. In an LCD display, theadditions of white can be made by increasing the transmittance of therespective color filters for the red, green, and blue subpixels. Theincrease may be accomplished by increasing the transmittances within theexisting filter wavelength ranges, or by broadening the spectral rangesthat the filters will pass (i.e., transmitting “less pure” red, green,and blue), or by a combination of these. Alternatively, a white subpixelmay be added to each pixel, or “open filter” regions surrounding thered, green, and/or blue subpixels may be added to each pixel. Inaddition to the addition of white to the image of 14B, the powerprovided to the backlight of the LCD display has been reduced. Thisresults in an image that has approximately the same brightness as theimage of FIG. 14A, but at a significantly reduced power consumption.

However, it also can be seen that the addition of the respective amountsof white to the red, green, and blue subpixels has resulted in the imageof FIG. 14B being significantly desaturated. FIG. 14C depicts an imagethat has been further processed using one of the Applicants' 3DLUTs. Thecolor restoration is significant over the unprocessed image in FIG. 14B,illustrating the effectiveness of the Applicants' method. The overallcolorfulness of the image has been restored to a level comparable tothat of FIG. 14A, but at approximately the reduced power consumptionlevel of FIG. 14B. The overall power savings of this example is 51%, asindicated in TABLE 1. Hence by the use of the Applicants' 3DLUTs andchromatic adaptation, power consumption by the display is reduced whilemaintaining high quality color images.

Although the above analysis is for a backlight LCD display with a whiteLED light source, the instant methods applies to any display technologysuch as OLED, laser, RGB LED display, and displays with inherently largeinitial color gamuts such as those with nanotechnology color primaries.In each case the 3DLUTs will need to be modified depending on thedisplay physics, but results similar to those above will be achieved.Additionally, the larger the starting color gamut of the display, morepower savings can be achieved. The above example is based upon adding astatic amount of white for each pixel, so the loss in color saturationis throughout the color space.

From a comparison of colorfulness values in TABLE 1 before and aftercolor processing with the Applicants' 3DLUTs, it can be seen that theapplicants' method can produce the same overall CieCam02 perceptualcolorfulness volume for a color gamut that is greater than 50% smallerin CieLuv. This is one way in which power savings can be achieved and italso shows that by using the Applicants' method, the cost of anacceptable display can be reduced significantly with a much smallercolor gamut.

Examples of Display Power Savings

1.) Summary of Examples—General Principles

Certain aspects of the Applicants' methods to increase the efficiencyand thus reduce the power consumption of a color display will now bedescribed in further detail by way of examples. In the followingdisclosure, it is noted that all examples and their analyses were madeby image simulation and visual analysis. Although no actual displayhardware was modified, the Applicants believe that the analysesdisclosed herein result in the same conclusions as would be reached ifmodifications were made to a display and then display image data wereobtained.

In the following examples for an LCD display, an adaptive amount ofwhite was added to each pixel of a given image to increase brightness.The average brightness increase for each image was then calculated andused to estimate the power savings for that image. The added white wasassumed to be provided by a 4^(th) , controllable pixel that was eitherproduced by a 4^(th) clear filter with full transmittance of thebacklight white, or a time-dependent white light segment being a portionof the exposure time of a pixel, or possibly a separate white lightsource that is added to the pixel exposure. Adding a white sub-pixel inimage displays has been disclosed in a variety of technical papersincluding “Review paper; Multi-primary-color display: The latesttechnologies and their benefits ” Teragawa, et all, SID20.1.1.

In one aspect of the power savings invention, the Applicants' 3DLUTs areused to restore color as shown by the change in the image of FIG. 14B toFIG. 14C. Additionally, in certain embodiments, a unique pixel dependentaddition of white that is a function of the original pixel saturation isapplied to the image. In certain embodiments, a Gaussian function ofpixel saturation may be used to calculate the pixel-dependent amount ofadded white, with less white added as the original pixel saturationincreases. Advantageously, this pixel dependent addition of white helpsto preserve the original color gamut of the display, and onlydesaturates the pixels that have lower original color saturation.

The calculation of this Gaussian white value can be implemented in realtime on images so that the power savings can be achieved image-by-imageusing a single one of the Applicants' 3DLUTs. The Applicants have foundthat grey scale images with the least amount of color have the mostpower savings, because the amount of added white and brightness increasewas highest for grey, neutral pixels. The maximum amount of added whiteat the peak of the Gaussian function and the width of the Gaussianfunction in color saturation were varied in the analysis to study andcompare the resulting image quality and power savings. Two differentlevels of maximum white were examined: 1.0 and 2.0, with 1.0 meaning themaximum white added was equivalent to the white attained from thecombination of the red, blue, and green primary colors of the display;and 2.0 meaning the added white was two times greater than the whiteattained from the combination of the red, blue, and green primary colorsof the display.

It will be apparent that other levels of maximum white may be suitablefor achieving a desired level of power savings. It is also to beunderstood that other decreasing functions can be used to define thepixel-dependent amount of added white, which achieve satisfactory powersavings results. The present invention is not limited only to Gaussianfunctions.

Since a clear filter of an LCD display will have much highertransmissivity than the combined red, green, and blue filters, it isnoted that in certain embodiments, this maximum added white may beincreased even beyond the 2.0 factor. Higher factors may be advantageouswhen actual hardware displays are modified to implement this adaptivewhite image display and power saving method.

2.) Summary of Examples—Images Analyzed

Ten images were analyzed in this study. The range of power savingsachieved was from 36%-50% for an added white maximum of 1.0; and 53%-67%with an average of 64% for the ten images for an added white maximum of2.0. As will be seen from the image comparisons to be presentedsubsequently, no loss in total color gamut or visual color qualityoccurred after application of the Applicants' 3DLUTs and chromaticadaptation. This illustrates the value of using the instant adaptivewhite method over the previously referenced static white method, whichhad a power savings of 51%. It is also in agreement with the Applicants'previously disclosed apparatus and methods for displaying a color image,which provide the most increase in color saturation for low andmid-level saturated pixels to improve the average visual color ofimages. It is also consistent with image statistics in general, whichindicate that pixels of images predominantly have low-to-mid level colorsaturation. It can be seen that the Applicants' power savings method issupported by image statistics in that advantageously, the greatestbrightness increase and the greatest power savings occur in the mostcommon pixels.

It is noted that the Applicants' adaptive white method may be applied toany display that has a controllable 4^(th) white sub pixel with time orarea segmentation and any starting color gamut. Static or mobile liquidcrystal displays, plasma displays, OLED displays, and projectors withtime dependent control of primary colors are all display technologies inwhich the Applicants' power savings methods may be implemented. In termsof commercial products, and without limitation to the following list,the Applicants' power savings methods may be applied to televisions,computers, tablets, cell phones and games.

3.) Image and Color Primary Modification Methods

The steps of modification of the primary colors and for subsequent imagemodification will now be described. These steps were used in theexamples described subsequently, and are also applicable in general topower savings in the above-recited displays.

-   -   (a) The color primaries of the display are to be defined. The        display may have sRGB color primaries.    -   (b) An amount of white is added to each RGB pixel to increase        the brightness, T₂ below. The amount of white is adaptive, i.e.,        for any given pixel, the amount is dependent upon the level of        color saturation of that pixel. The amount of white added as a        function of color saturation may be determined from the        following equations:

T₁ = min (rgb) T₂ = z × (T₁)$z = {A\;{\mathbb{e}}^{- {({\frac{{({x - x_{w}})}^{2}}{2\sigma_{x}^{2}} + \frac{{({y - y_{w}})}^{2}}{2\sigma_{y}^{2}}})}}}$

where

“min(rgb)” means the minimum common value; for example, in the precedingexample where the additions of white were 10% white to blue, 20% whiteto red, and 30% white to green, “min(rgb)” is 10%.

x and y are the pixel chromaticity values;

x_(w) and y_(w) are the chromaticities of the white point;

A is a chosen parameter that specifies the maximum white added at thewhite point; and

σ_(x) and σ_(y) are chosen parameters that specify the width of theGaussian in chromaticity.

The two dimensional Gaussian z function is shown in FIG. 15 for a whitepoint (x_(w), y_(w)) of (0.0,0.0) and σ_(x) and σ_(y) values of 1.0. Thevalue of this function at a σ value radius of 1.0 is 0.367 and at a σvalue radius of 2.0 is 0.0183, illustrating the decrease in added whiteas the pixel value moves out in color saturation.

-   -   (c) The dependence of the total gamut volume in CieLUV color        space, the relative luminance increase and the power savings        were analyzed for various Gaussian σ values and plotted in        FIG. 16. This FIG. shows that the total color gamut is        preserved, as indicated by gamut line 502. It can also be seen        that most of the added luminance 504 and power savings 506 are        achieved with a log 10 σ value of −0.6, which is equal to a σ        value of 0.25.

This is a noteworthy σ value because it is a chromaticity radius that iswell inside the sRGB chromaticity plot as shown in FIG. 17. The circle508 within the color gamut 510 indicates that that the majority of thewhite is added to lower color saturated pixels that are closer to thedisplay white point (D65 for sRGB), and that the more saturated pixelhave no added white so the maximum color saturated pixels are notdesaturated. This is highly consistent with the Applicants' colorprocessing methods in general because it is most effective in increasingthe low and mid-level pixel color saturation to raise the average colorsaturation of images.

-   -   (d) Grey-level images were created showing the amount of        brightness increase for each pixel and the average brightness        increase from those images was used to calculate the average        image power savings. The average image power savings are shown        for two exemplary images in TABLE 2.

TABLE 2 Power savings for two exemplary images for maximum added whitevalues of 1.0 and 2.0 and Gaussian chromaticity sigma value of 0.25PHOTOGRAPH SET POWER SAVINGS (%) FIG. NO. SUBJECT MATTER A = 1.0, σ =0.25 A = 2.0, σ = 0.25 19D Sea of Flowers 46 63 20D Snow 49 66

-   -   (e) The Applicants' 3DLUTs and chromatic adaptation were used to        restore the loss color saturation in the images to which white        was added. FIG. 18 shows the colorfulness measure with the        Applicants' color image processing (plot 512) and without (plot        514), illustrating the color restoration capability of matching        the colorfulness for a display with half the total color gamut.        The Applicants' color image processing includes lowering the        white point of an image to better match the brightness of white        and saturated colors in the image to help in color restoration.        Two exemplary images resulting from the Applicants' color image        processing are shown in FIGS. 19C and 20C and will be described        next in further detail. They represent the final brightness of        each pixel after power savings and the use of the Applicants'        3DLUTs and chromatic adaptation.

4.) Simulation Image Analysis and Colorfulness Measure

In one study, in order to judge the effectiveness of restoring the lostcolor saturation for the Applicants' power savings image processingmethod, ten experimental images were processed. In addition, acolorfulness measure was calculated that has been shown to be linearwith visual rankings across a large number of image types by the MunsellColor Science Laboratory (MCSL) in the Rochester Institute ofTechnology. This is an important factor in the analysis of the imagesbecause the chromaticity color saturation of image pixels are clearlybeing reduced; in order to achieve favorable perception by a viewer ofthe image, it is important for to replace that loss in colorchromaticity with a measure that is more related to how users see thecolor quality of the modified panels. The colorfulness measure used inthe analysis is not a color gamut area, but rather a statistical measureof how colorful a set of images would be to users throughout the fullcolor gamut volume. It has been shown to be a good measure of visualrankings with a nearly linear relationship from a long history ofempirical psychophysical data consolidated from many researchers in thefield and under many viewing conditions. This was developed by the CIEin CIECAM02 which in turn was based on the work of many people anddiscussed in Color Appearance Modeling for Color Management Systems, CIETechnical Conference 9-01, 2002 by Mark Fairchild and R W Hunt. It isalso known generally as a color appearance standard.

Using the Applicants' 3DLUTs that can boost the color saturation usingvisual models for midrange color saturation levels, this averagecolorfulness can be increased for any color gamut. Keys for thiscolorfulness measure are that it is linear with visual rankings and thatit covers all of a three-dimensional visual color space, not just aplanar slice showing primary vector projection boundaries. It isbelieved to much better represent an image viewer's color experience fora system than the chromaticity gamut boundary. The Applicants have foundthat including their color image processing for a small color gamutdisplay produces image results that are more colorful than a 100% largercolor gamut display.

5.) Gamut Mapping Method and Hue Preservation

The methods of these analyses can result in hue shifts in the resultantimages processed by the Applicants' methods unless the gamut mapping,increased color saturation processing and the addition of white toincrease brightness are carried out in a non-uniform hue color space.The gamut mapping component of the lack of hue preservation was reportedby Daly, et al in “Gamut Mapping in LCD backlight compensation”, 16^(th)Color Imaging Conference, May 31, 2011. Others who have attempted to useadded white for extra brightness have also faced hue shifts. This isbecause processing in XYZ chromaticity color space for added white canmodify hue significantly because the I hue path in XYZ chromaticitycolor space is not a straight line between the color primaries andbacklight white; in fact it is not a line at all but rather a curvedpath. For this reason all the gamut mapping, white addition and use ofthe Applicants' 3DLUTs to increase saturation were done in a uniform huecolor space known as IPT. This color space was first defined byFairchild and Ebner at the 1998 CIC Conference “Development and Testingof a color space (IPT), with Improved Hue Uniformity”. The Applicants'3DLUTs are defined in this IPT color space. This enables the lostcolorfulness from the addition of white to be restored withoutartifacts.

6.) Adaptive White Using a Gaussian Function of Pixel ColorSaturation—Image Results

Ten images were analyzed ranging from black-white text to highlycolorful images to illustrate the variation in power savings andresultant image quality. This was done both for a maximum added white atthe display white point of 1.0 and 2.0. The power savings are shown inFIG. 16. Two exemplary image sets from the ten images of the study areprovided in FIGS. 19A-19D and 20A-20D. In the image sets, only therestored image results for a maximum added white value of 2.0 are shownbecause they are visually equivalent to the restored images for amaximum added white value of 1.0.

There are sets of four images shown for each of the two exemplary imagesthat were among the ten included in the analysis: original images areshown in FIGS. 19A-20A; original plus added white without theApplicants' color image processing are shown in FIGS. 19B-20B; originalplus added white with the Applicants' color image processing are shownin FIGS. 19C-20C; and grey scale luminance images showing a luminanceincrease are shown in FIGS. 19D-20D. It can be seen in particular fromthe pair of original images 19A and 20A in comparison, respectively, tothe images of FIGS. 19C and 20C that include processing with theApplicants' 3DLUTs, that the latter images are highly saturated and ofsuperior image quality, while achieving the previously described powersavings.

7.) Additional Display Modifications and Examples

In certain embodiments, a display may be modified to reduce powerconsumption, or to make it brighter while using the same amount of powerbut also having a resulting smaller color gamut. The modification thatis made is dependent upon the type of display. As was describedpreviously, in a liquid crystal display, the red, green, and blue colorfilters may be changed to make them more transmissive, i.e. to allowthem to pass more light within a wavelength range, or a broader portionof the spectrum. Alternatively or additionally, the size of the colorfilters may be reduced to cover less area in a pixel, thereby allowingmore white light to pass. This may have the effect of having a fourthwhite subpixel, with red, green, and blue being the colored subpixels.Alternatively or additionally, during the duty cycle of operation of apixel, some of that duty cycle may be used to have all three primarycolors on to produce added white.

Alternatively or additionally, a white light source may be added that isindependent of the operation of the primary color light sources. For LEDdisplays, this may simply be adding a white LED to each pixel. For LCDdisplays having white backlights with color filters, this may be done byproviding a second light path that allows the primary color light topass through the filters, or that adds to the filtered light.

Alternatively or additionally, the modulator of the display may bemodified so it lets more primary color light through and out of thedisplay. This may be done at the expense of introducing color cross-talkto the red, green and blue color primaries, which may combine to changethe color gamut of the display. This may be addressed by displacing thecolor of the red primary toward green, for example, which will shrinkthe color gamut.

Alternatively or additionally, a material change in the display opticsmay be made that causes more light to be diverted outward through thedisplay modulator. Although this may result in some color crosstalk thatmakes the initial RGB color primaries less “pure.”

Alternatively or additionally, in an LCD display, the power of the whitebacklight may be increased beyond the linearity of the LCD modulator sothat color primaries are changed and their brightness is increased.

The above descriptions of modifications to a display for the statedpurposes are meant to be exemplary and not limiting. Other modificationsto a display are contemplated.

The effects of these display modifications may be compensated for byusing three dimensional color tables as described previously and as willnow be further illustrated using certain additional examples.

FIGS. 9 and 10 depict one option for solving the problem of displayinghigh quality images on a portable display device over an extended periodof time within the constraint of battery life. More specifically, FIG. 9is a graphical representation of a chromaticity diagram, including afirst color gamut transformation that enables a reduction in powerconsumption by a display, in accordance with the present invention. Ifthe display is powered by a battery, the life of the battery is extendedas a result of the reduction in power consumption.

Referring first to FIG. 9, the color gamut 430 of a display device isshown. In a first step to reduce the power used by the display, a colorgamut transformation may be performed by changing the primary colors431, 432, and 433 of the color gamut 430 to new primary colors 441, 442,and 443, which define a new color gamut 440. In certain embodiments, thetransformation may be performed by “rotating” the color gamut 430 asindicated by arcuate arrows 434 to produce the new color gamut 440. Incertain embodiments, the color gamut 430 may be rotated around the whitepoint, which may be D65. Transforming the color gamut 430 to color gamut440 having primary colors 441, 442, and 443 has the effect of increasingthe white point of the new color gamut 440 when an optimal selection ofprimary colors 441, 442, and 443 is made.

It is noted that in general, when a color gamut is rotated, thewavelengths that are transmitted are expanded. For example, if red isdisplaced toward blue via clockwise rotation, then blue is added to thered primary color, making it brighter. Likewise for the blue and greenprimary colors. To further illustrate this point, it is noted that the“midpoint” colors cyan, magenta and yellow are brighter than red, greenand blue. Additionally, if the white point is increased by rotating thecolor gamut, the color of the white may be changed, i.e. tinted. TheApplicants' methods described herein accommodate such a change,correcting the color of an image to the desired appearance.

In addition to this color transformation, the transmittance of thescreen of the display device (such as an LCD display) is increased. Forthe display of a given image, a color correction is applied to inputimage data in accordance with the methods previously described in thisspecification. The color correction may be performed according to twodifferent procedures. In a first procedure, the extra brightness of thedisplay resulting from the increase in screen transmittance ismaintained, and visual color adaptation using an at leastthree-dimensional lookup table is performed to adjust the output colordata to the new white point of the new color gamut 440. An image isdisplayed from the color-corrected output image data.

If the new white point of the new color gamut 440 is too different fromD65 white, the resulting colors within the new color gamut may appeartinted to an observer of the display. In such circumstances, a secondprocedure for color correction may be used. In this second procedure thewhite point of the new color gamut 440 is mapped to the original D65standard. This will result in the loss of some brightness, but not to adegree that is perceivable by an observer. As in the first procedure,color correction is performed using an at least three-dimensional lookuptable to adjust the output color data to correspond to the mapping ofthe white point of the new color gamut 440 to the D65 white. The outputcolor data is shifted so that to an observer of the display, it lookscorrect with respect to the new white point. For example, if the newwhite point has a yellowish tint, then the colors are shiftedaccordingly toward yellow (i.e. a combination of red and green that isequivalent to yellow).

Referring to FIG. 10, these options are shown in a 2D slice of thethree-dimensional color volumes in the color gamut transformationdepicted in FIG. 9. The slice depicts the red and green primary colors.The color values of the original color gamut 430 are bounded by the fourline segments 435-438. The green primary color G is at the intersectionof line segments 435 and 436, and the red primary color R is at theintersection of line segments 437 and 438. The white point W, which maybe D65 white, is at the intersection of line segments 436 and 437.

The color gamut 430 is transformed to color gamut 440 as shown in FIG.9, and the brightness of the display is increased as described above,resulting in new color values of the new color gamut 440. The new colorvalues include a new maximum brightness white point W+, with the whitepoint W of the original color gamut 430 being among the new colorvalues. The new color values also include a new red primary color at R′and a new green primary color G+, which are brighter due in part to theincrease of the brightness of the display. In an LCD display, theincreased brightness of the respective primary colors may be attained byusing more transmissive filters for filtering the backlight of thedisplay, as described previously.

With the increased brightness of the new white point W+, the power tothe backlight of the display may be reduced, thereby reducing displaypower consumption. When this is done, a new red primary color R+results, and the new color values are bounded by line segments 445-448.It is noted that a similar effect occurs for the green primary color G,which is not shown for the sake of simplicity of illustration. Thepreviously described first and second procedures may be performed toeffect a color correction.

In the first procedure, the maximum brightness white W+ may bemaintained for use in the display, with visual color adaptation using anat least three-dimensional lookup table being performed to adjust theoutput color data to the new white point W+. New color values in region442 are added to the color gamut of the display.

In the second procedure, color mapping is done to map the maximum whiteW+ back to the original D65 white W. Additionally, a three-dimensionallookup table is used to obtain a smooth color mapping to the originalD65 white with the new R+ and G+ primary colors, as indicated by linesegments 441 and 444.

It is noted for both of the above first and second procedures, a smallregion of color gamut 449 is lost in these transformations. This loss ofcolor gamut is not significant with respect to perception by a user ofthe display.

FIG. 11 is a graphical representation of the chromaticity diagram ofFIG. 9, including a second color gamut transformation that enables areduction in power consumption by a display, in accordance with thepresent invention. A color gamut 450 of a display is shown.Additionally, regions 451, 453, and 455 near respective primary colors452, 454, and 456 are shown as denoted by ellipses.

In a color transformation to a new color gamut, new primary colors areselected, wherein each of the new primary colors is selected within theregions 451, 453, and 455. For example, a new color gamut 460 is showncomprising new primary colors 462, 464, and 466. The new primary colorsare chosen for increased efficiency in that brighter new primary colorsare chosen. By choosing such brighter primary colors, the power to thedisplay may be reduced. The regions 451, 453, and 455 may be larger thanas shown in FIG. 11, or shaped differently. Additionally, moredesaturation of a single primary color, such as blue, may be used. Thedesaturated color is brighter, and hence the power to the display may bereduced.

As in the previous embodiment depicted in FIG. 10, if the new colorgamut 450 is close to enclosing the RGB values of sRGB that isbeneficial. More saturation is also beneficial, but may occur at theexpense of efficiency. In other words, there is a tradeoff betweenpreserving as much saturation as possible while also obtaining thebrightest primary colors, which enable the reduction in powerconsumption. The Applicants' adaptive method using 3DLUTs is effectivebecause it does not add white to the most saturated pixels, therebypreserving the primary colors and overall color gamut, while adding agreater amount of white to the average pixel to maintain brightnesswhile reducing power consumption.

Any color values within the new color gamut 460 may be corrected by theuse of three dimensional lookup tables according to the methodsdescribed previously in this specification. The methods described andshown in FIG. 10 with regard to a new white point and increasedbrightness are also applicable to the color gamut 460. As describedpreviously, because the primary colors of the new color gamut 460 arebrighter, the power to the display may be reduced while maintaining theoriginal brightness of the display.

FIG. 12 is a two-dimensional “slice” of the three-dimensional colorvolume resulting from a third color gamut transformation in accordancewith the present invention. In this transformation, the transmittance ofone primary color in the display is increased. In other words thebrightness of the one primary color is increased. In an LCD display, theincreased brightness of the primary color may be attained by using amore transmissive filter for filtering the backlight of the display.

Referring to FIG. 12, and in the example depicted therein, the originalcolor values of the color gamut of the display are bounded by linesegments 471, 472, 473, and 474, with a D65 white point. The brightnessof the red primary color is increased from R to R′ as indicated by arrow475. This results in a new white point 476 that is brighter but appearsto have a reddish tint to an observer of the display.

One of several color corrections may be performed at this point. In afirst correction, the backlight of the display is reduced, therebyreducing the power of the display. Additionally, chromatic adaptationfrom a visual model of the human visual system is used to produce an atleast three-dimensional lookup table, which is used to adjust the outputcolor data to the new white point 477 of the new color gamut of thedisplay. The “slice” of the new color gamut produced by this colorcorrection is bounded by line segments 478, 479, 480, and 481, and thered and green primary colors are R″ and G′.

In a second color correction, the original D65 white may be mapped to anew dimmer white 482 as indicated by arrow 483. In this colorcorrection, the new maximum brightness may remain as a reddish whitewhich may be similar to reducing the white point, i.e., the maximum redbrightness may appear very red. A slightly reddish white for the whitepoint is acceptable to an observer with the use of the Applicants' 3DLUTwith chromatic adaptation to shift all the colors toward red, therebymaintaining the visual color relationships of the image as perceived bythe observer.

In a third color correction, a combination of the above first and secondcolor corrections may be performed.

FIG. 13 is a graphical representation of an exemplary set of colorgamuts in which the saturation of primary colors is reduced, whichenable a reduction in power consumption by a display, in accordance withthe present invention.

For reference, the standard sRGB color gamut 490 is shown in coarsedotted line format and having respective red, green, and blue primarycolors 491, 492, and 493. A color display that has an sRGB color gamut490 may be provided. The power consumed by the display may be reduced byreducing the saturation and/or brightness of the primary colors 491-493,as indicated by color gamut 494 in solid line. However, the overallaesthetic appeal of images on the display will be less satisfactory. Asa first alternative, the saturation and/or brightness of the blue andthe red primary colors may be reduced, and the sRGB green primary colormaintained. This results in color gamut 495 shown in fine dotted line,with a significant reduction in power being attained. To obtain colorthat is satisfactory to an observer of the display, as describedpreviously in this specification, color correction is performed using anat least three-dimensional lookup table to adjust the output color datato correspond to the mapping of the white point of the new color gamut494. The three-dimensional lookup table may be determined from a visualmodel of the human visual system including chromatic adaptation of theHVS.

As a second alternative, the saturation and/or brightness of only theblue primary color may be reduced, and the sRGB green and red primarycolors maintained. This results in color gamut 496 shown in mediumdotted line, with a significant reduction in power still being attained.To obtain color that is satisfactory to an observer, color correction isperformed as recited above for color gamut 495.

In all of the above alternatives, the reduction in saturation isachieved by an increase in brightness by the methods described herein.Subsequently, the power to the display is reduced, thereby reducing thebrightness back to approximately the original levels. The Applicants'3DLUTs are used to adaptively restore the desired levels of saturationto the individual pixels, resulting in an image that is perceived to beof comparable colorfulness to that of the original image. The instantmethods are applicable to all types of image displays and imageprojectors.

It is, therefore, apparent that there has been provided, in accordancewith the present invention, methods and devices for producing a colorimage. Having thus described the basic concept of the invention, it willbe rather apparent to those skilled in the art that the foregoingdetailed disclosure is intended to be presented by way of example only,and is not limiting. Various alterations, improvements, andmodifications will occur and are intended to those skilled in the art,though not expressly stated herein. These alterations, improvements, andmodifications are intended to be suggested hereby, and are within thespirit and scope of the invention. Additionally, the recited order ofprocessing elements or sequences, or the use of numbers, letters, orother designations therefore, is not intended to limit the claimedprocesses to any order except as may be specified in the claims.Accordingly, the invention is limited only by the following claims andequivalents thereto.

We claim:
 1. A method of producing a color image using a display devicecomprised of pixels comprising red, green and blue primary colorsubpixels, the method comprising: a) increasing the brightness of theimage relative to a base level by changing display device primarycolors, thereby changing the color gamut of the display device; b)decreasing power to the display device to reduce the brightness of theimage; c) restoring color to the image to approximately the base levelby modifying image pixel data using a three-dimensional lookup table toproduce output image pixel data; and d) communicating the output imagepixel data to the display device to produce the color image; whereindecreasing power to the display device reduces the brightness of theimage to a level brighter than the base level.
 2. The method of claim 1,further comprising adding white to the image to reduce the color gamutand increase the brightness of the image.
 3. The method of claim 2,wherein white is added by a white subpixel.
 4. The method of claim 2,wherein white is added to at least one of the primary color subpixels.5. The method of claim 2, wherein white is added to two of the threeprimary color subpixels.
 6. The method of claim 2 wherein white is addedto the three primary color subpixels.
 7. The method of claim 6, whereinwhite is added to the three primary color subpixels in unequal amounts.8. The method of claim 6 wherein white is added adaptively according toan algorithm by which the amount of white added decreases withincreasing color saturation.
 9. The method of claim 8, wherein thealgorithm by which the amount of white added decreases with increasingcolor saturation includes a Gaussian function that defines the decreasein white with increasing color saturation.
 10. The method of claim 8,wherein the algorithm is used to determine the values in thethree-dimensional lookup table.
 11. The method of claim 8, wherein aplurality of images are produced using the display device, and whereinthe algorithm includes determining the amount of white to add to eachimage pixel and the amount of the decrease in power for each imagepixel.
 12. The method of claim 8, wherein the algorithm includesdetermining the amount of white added for each individual pixel.
 13. Themethod of claim 12, wherein the algorithm includes determining theamount of white added for each individual red, green and blue primarycolor subpixel.
 14. The method of claim 2, wherein white is added to thesubpixels during a portion of a pixel exposure time.
 15. The method ofclaim 2, wherein white is added from a second source that is separatefrom a first source providing the red, green and blue primary colorsubpixels.
 16. The method of claim 15, wherein the display device is anLCD display device comprising a first backlight as the first source, anda second backlight as the second source.
 17. The method of claim 2wherein the display device is an LCD display device, and the white isadded to each pixel by a white subpixel.
 18. The method of claim 1,wherein the display device is one of an LCD display device, an LEDdisplay device, an OLED display device, a plasma display device, and aDMD projector.
 19. The method of claim 1, wherein decreasing power tothe display device reduces the brightness of the image to approximatelythe base level.
 20. The method of claim 1, wherein memory colors arepreserved in the color image.
 21. The method of claim 1, whereinrestoring the color of the image includes correcting the white point ofthe display device to a white point of a color standard.
 22. The methodof claim 1, wherein the restoring the color of the image is performed inthe IPT uniform hue color space.
 23. The method of claim 1, wherein thevalues in the three-dimensional look-up table are determined by using avisual model of the human visual system.
 24. The method of claim 23,wherein the values in the three-dimensional look-up table are determinedusing chromatic adaptation of the human visual system.
 25. A method ofproducing a color image using a display device comprised of pixelscomprising red, green and blue primary color subpixels, the methodcomprising: a) increasing the brightness and reducing the colorsaturation of the image relative to a base level by adding white to theimage pixels; b) decreasing power to the display device to reduce thebrightness of the image; c) restoring the color saturation of the imageto approximately the base level by modifying image pixel data using athree-dimensional lookup table to produce output image pixel data; andd) communicating the output image pixel data to the display device toproduce the color image; wherein decreasing power to the display devicereduces the brightness of the image to a level brighter than the baselevel.
 26. The method of claim 25, wherein white is added by a whitesubpixel.
 27. The method of claim 25, wherein white is added to at leastone of the primary color subpixels.
 28. The method of claim 25, whereinwhite is added to two of the three primary color subpixels.
 29. Themethod of claim 25 wherein white is added to the three primary colorsubpixels.
 30. The method of claim 29, wherein white is added to thethree primary color subpixels in unequal amounts.
 31. The method ofclaim 29 wherein white is added adaptively according to an algorithm bywhich the amount of white added decreases with increasing colorsaturation.
 32. The method of claim 31, wherein the algorithm by whichthe amount of white added decreases with increasing color saturationincludes a Gaussian function that defines the decrease in white withincreasing color saturation.
 33. The method of claim 31, wherein thealgorithm is used to determine the values in the three-dimensionallookup table.
 34. The method of claim 31, wherein a plurality of imagesare produced using the display device, and wherein the algorithmincludes determining the amount of white to add to each image pixel andthe amount of the decrease in power for each image.
 35. The method ofclaim 31, wherein the algorithm includes determining the amount of whiteadded for each individual pixel.
 36. The method of claim 35, wherein thealgorithm includes determining the amount of white added for eachindividual red, green and blue primary color subpixel.
 37. The method ofclaim 25, wherein white is added to the subpixels during a portion of apixel exposure time.
 38. The method of claim 25, wherein white is addedfrom a second source that is separate from a first source providing thered, green and blue primary color subpixels.
 39. The method of claim 38,wherein the display device is an LCD display device comprising a firstbacklight as the first source, and a second backlight as the secondsource.
 40. The method of claim 25 wherein the display device is an LCDdisplay device, and the white is added to each pixel by a whitesubpixel.
 41. The method of claim 25, wherein the display device is oneof an LCD display device, an LED display device, an OLED display device,a plasma display device, and a DMD projector.
 42. The method of claim25, wherein decreasing power to the display device reduces thebrightness of the image to approximately the base level.
 43. The methodof claim 25, wherein the restoring the color of the image is performedin the IPT uniform hue color space.
 44. The method of claim 25, whereinthe values in the three-dimensional look-up table are determined byusing a visual model of the human visual system.
 45. The method of claim44, wherein the values in the three-dimensional look-up table aredetermined using chromatic adaptation of the human visual system. 46.The method of claim 25, wherein memory colors are preserved in the colorimage.
 47. The method of claim 25, wherein restoring the color of theimage includes correcting the white point of the display device to awhite point of a color standard.